Invenzzia... in English - OPT2

OPT 2.0.0-dev4

Zyx — Sun, 02 Mar 2008 11:28:00 +0100

There is a new Open Power Template development version available to download. The project begins to gel, as the main parser class is almost completed. Inspite of the output cache system mentioned earlier, I've implemented new instructions and completed the snippets. It means that all the template inheritance should work now. I hope the library will be usable by the end of this month. The real projects are really necessary to detect possible bugs and various shortcomings.

In snippets, we have a new possibility to insert the parent content using the opt:parent tag:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<opt:extend file="test_inherited_a.tpl">
	<opt:snippet name="header">
		<h1>Webmaster Of Puppets</h1>
		<p>Here is the parent content:</p>
		<opt:parent/>
		<p>End of the parent content</p>
	</opt:snippet>
</opt:extend>

In this case, if the test_inherited_a.tpl contains another snippet with the same name, its content will be put in the place of opt:parent. Such nesting works much deeper. For example, let's say that snippet also uses this tag. OPT will try to locate another parent and so on. It is even possible to put the default content of the opt:insert tag, which is used to place the snippets. The code contains also a simple code to detect the infinite recursion. If the parser finds it, it returns an exception and the programmer sees a nice, red error message :).

During the using of the new OPT, the new debug console should be also useful. It has been extended, compared to the similar option in OPT 1.1.x. The most significant change is splitting the list of templates to the smaller ones which show different information:

Executed templates
Currently compiled templates
Templates read from cache.
Inherited templates - it shows all the inheritance chains, as well, as the information whether they have been recompiled or not.

Moreover, the console displays all the warnings. They are no longer put onto the screen, but available only with the debug console or error messages, if we set the proper debug level.

The performance issue also looks very good. The total size of opt.class.php file is about 25 KB, compared to 32 KB in the OPT 1.1.x. Currently, its size influences the performance much, as PHP must load the parser and compile it to the internal bytecode. Of course, if we parse a lot of templates, it won't be so visible, however I'm quite pleased of the effect.

The package can be downloaded from here.

The main class almost finished

Zyx — Fri, 22 Feb 2008 08:59:00 +0100

The result of the latest works on the OPT source code is complete opt.class.php file which contains (as some of the readers may know) the main parser class. I must admit that this file was more demanding that the compiler. In the second case we can afford with quite a lot. Just an idea for the algorithm and prediction if and how it will influence the rest of the code. In the case of the main class, we must pay attention to the performance. It must contain everything that is run permanently and it must be fast and small. But the number of features is getting bigger and bigger.

The new development version will not be compatible with the previous ones when it comes to the programming interface. To implement the output cache system, I had to rebuild almost all of the output system. In the 1.x.x series, the concept of the output system did not exist. Either the result will be seen on the screen or put into a variable, it depended on the right method choice. There were some tricky connections which redirected everything to the right place, however it was not well designed and a bit messy. Well, such things simply must happen, if we start with keeping the similarities to Smarty, and I have to remind that in the very early times, it was one of the OPT goals. In OPTv2, there is a well-declared ioptOutput interface allowing the programmer to code the output processing on his own. It is also used in the implementation of output cache system. How - it is explained below.

So far, there have been two output systems: optNetOutput sending everything on the screen and checking, how many templates we are going to send in the XML mode. The second one was optReturnOutput, which returned the result. I added a possibility to decorate them with something else and I created optCachedOutput. Its only goal is to process the cache. The features are the same as in the previous versions, but the usage is a bit different and unfortunately, the initialization is more complicated:

$cache = new optCachedOutput('./cache/'); // this is the cache, here we configure it
$tpl -> register(OPT_OUTPUT, 'my_cache', $cache);
$tpl -> decorate(OPT_OUT_SCREEN, 'my_cache');
$tpl -> decorate(OPT_OUT_RETURN, 'my_cache');

if(!$cache -> isCached('template.tpl', 'some_id'))
{
  // data generation
}

$cache -> setCache(5, 'some_id'); // turn the cache on for 5 seconds
$tpl -> parse('template.tpl', 'my_cache');

I'll try to shorten it and make simpler the first lines of the code, because I don't like them, too. Moreover, I must check if I've made the decoration in the right way (according to the logic).

The support for plugins is not fully completed yet. The only ready part is placed in opt.class.php file. The rest is moved somewhere else, which should improve the performance. Like in OPT 1.1.3, there will be a possibility to set many plugin directories, but the parser will not require them to be writeable. There were thrown some files describing the content and allowing to include everything without scanning all the directory. As we know, the more includes, the less performance and some people can produce more plugins than all the files in my framework :). The new OPT will try to link all these files into one, mega-file containing everything we need. By the way, I'll have also a new algorithm for the OPT Toolset sequel, which will be used now for all the libraries, not only OPT.

The last issue concerns the available function set for the templates. Stormfly has recently written on his blog that OPT 1.1.x it was quite senseless and in fact, he was right. I took a look, what Smarty offers and it turned out that it has some nice and useful functions. The first list looks like that:

spacify - puts a string between every two neighbour characters in the text.
indent - makes an indentation in the text.
truncate - reduces the text to the specified length (allowing not to splitting the words)
strip - reduces all the white character groups to a single character
capitalize - equivalent to ucfirst()
upper - equivalent to strtoupper()
lower - equivalent to strtolower()
countWords - equivalent to str_word_count()
countChars - equivalent to strlen()
money - equivalent to money_format(), but the settings can be written in the OPT configuration.
number - equivalent to number_format(), but the settings can be written in the OPT configuration.
wordWrap - equivalent to word_wrap()
replace - equivalent to str_replace()
regexReplace - equivalent to preg_replace(), but the parameter order is changed.
date - equivalent to date() :)
firstof - returns the first non-empty element from the specified ones.
nl2br - equivalent to nl2br() :)
array - creating arrays on the template side
build - it was called "apply()" in the previous versions, but I still don't know, whether I keep this name or not.

The set contains no escaping functions, because the compiler is able to put them on his own in most of the places. There will be a possibility to escape something manually, but it won't be done through functions. Anyway, the point is the compiler must know, what is escaped. Otherwise it may add additional function call and this would produce a complete mess on the screen.

To sum up, I've written so much functionality that I haven't tested all of them yet. I don't know if something works, however I don't expect any revelations. Many of the algorithms are reimplementations of the same code from OPT 1.1.x. The next dev version will be available in the nearest days.

Some statistics for the end:

- opt.class.php - 24 KB and still growing. In OPT 1.1.x, this file was 8 KB bigger, but offered less functionality.
- opt.compiler.php - 65 KB and still growing.
- opt.instructions.php - 25 KB and still growing.

OPT 2.0.0-dev3

Zyx — Fri, 15 Feb 2008 13:26:00 +0100

According to the my announcement from the last week, the development versions are going to appear much faster than before. This is the first of such events. The third dev of OPT 2.0.0 is ready to download. This time, I've paid much attention to the attribute processing. They can do a lot of interesting things with the tags they've been added to. The earl implementation has been ready so far - there were even examples showing the "opt:section" working. But to satisfy both me and probably other programmers, I had to extend it.

Here are some examples showing, what can be done now. Let's start from something simple. In OPT, we use curly brackets to put expressions into a text, like:

text text text {$block} text text text

There is also opt:put instruction, which does the same for the first time we see it:

text text text <opt:put value="$block"/> text text text

However, it is an instruction, not curly brackets. It means we can make some nice things with it using attributes. Let's say we want to display something like "Value 1 / Value 2 / Value 3". We will use opt:put, a section and a separator:

<p><opt:put value="$section.value" opt:section="section" str:separator=" / "/></p>

As we see, such section affects also the tag which it was linked to and we can put the values of section elements in the attributes. The separator accepts expressions by default, but if we don't like writing separator="'something'" (double, and then a single quote), we can just switch the namespace to "str".

The implementation of reversed quotes is also ready. Compared to the previous versions, they change their function. Now it is the programmer who can give them a meaning and program them to do what he wants. It can be done by writing a simple function and giving its name to the parser. A sample use is an elegant implementation of language system a'la gettext:

<p>{`Some useful information...`}</p>
<p>{`Author:`} {$author}</p>

This is the last of the changes. As we have an XML compiler, we need some special instructions that allow to manage the tree directly, for example by adding an attribute with a variable name. OPT is going to provide two instructions for this task: opt:tag and opt:attribute. The second one has been already added to the newest dev. We can use it to read the attribute name from the block:

<div>
  <opt:attribute name="$some_name" value="$some_value"/>
  body...
</div>

This is not the end - connected with sections, we get a powerful tool to add a lot of new attributes:

<div>
  <opt:attribute name="$attr.name" value="$attr.value" opt:section="attr"/>
  body...
</div>

If we send the following table to this template: array(0 => array('name' => 'class', 'value' => 'dude'), array('name' => 'id', 'value' => 'lol')), the result will be:

<div class="dude" id="lol">
  body...
</div>

Unfortunately, the English manual is not started yet, but it will be ready before the first stable release.

Download: http://www.invenzzia.org/download/opt-2.0.0-dev3.tar.gz

New dev

Zyx — Sat, 09 Feb 2008 11:02:00 +0100

The website now has the "Download" page. In it, you can find a new development version of Open Power Template. It also contains a simple script "docgen" which we use for automated generation of the DocBook manuals. The new development versions should appear now much more often than before. In case of many new features, they might appear even every 2-3 days. The reason why this was impossible in the past was the time. The process was not automated and it takes a while to prepare such archive (for example - it takes a couple of hours to publish new OPT 1.x version on the Internet).

The pack contains everything I've implemented for last month:

Nearly completed sections.
Separators.
"For" and "Foreach" loops.
Mostly completed template inheritance.
HTTP header management.
Partially completed prolog parser.
Some fixes in the compiler.

The "/dev" directory contains some scripts that are used to test the new features. There are also unit tests: a complete set to test the expression parser and a part of the API test. There will be also a one to process the instructions, however the whitespace characters cause many problems, when it comes to prepare the files. If we try to compare the generated content with the expected result, we have to watch out on every space or line break. Otherwise the test fails, so you must be very patient to write a simple, stupid test. But I think this issue will be solved. To work, the tests require "phpUnit 3" package available in PEAR.

Template inheritance

Zyx — Sat, 02 Feb 2008 17:03:00 +0100

Unsually, the final HTML code of the website is composed of several smaller templates containing smaller pieces of it. There are many techniques of combining them. In PHP template engines, the most popular are two forms: manual execution of the templates from the script side and the include directive. Similar solution can be found in OPT 1.x, however in the new version, it should be forgotten - unless we use the quirks mode, the parser does not let us to call the parse() method more than once. In the beginning, I planned to replace it by extending inclusion with "sequences" - a groups of templates to include with their own variables and cache settings. However, one day I was asked on my blog by Coldpeer, whether OPT will support template inheritance. I googled for a while, I looked at it and thought it might be cool. That's how the implementation began.

As far as I know, template inheritance appeared for the first time in the Python framework, Django. The idea is taken from the object oriented programming, where we can extend classes by adding new methods, fields and overwriting the existing ones. Here it looks similar. We have a main template where we mark some characteristic places and fill them with a default content. Another template extends the main one and is able to overwrite that places with its own content. There can go the next template, and next...

I was quite surprised that a lot of required functionality already exists in OPT 1.x - the bind and insert instructions allowed to move whole template parts somewhere else (as well as their functionality, not only the result, like in Smarty's capture), even if the destination was in the other template. Unfortunately, there was no proper detection, whether one of the files was modified, so unsually the parser decided that the main part needs to be compiled, but the template with snippet definitions - does not. As a result, the places remained empty, although they should not. The development of such system for OPT 2.0 was quite complicated. You do not have to check just the template you've called, whether it is modified. You must also look at all the templates it inherits. Yesterday, I decided to solve this problem definietly and a couple of hours ago everything seemed to work properly:

The base template (test_inherited_a.tpl):

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" lang="en_US" xml:lang="en_US">
 <head>
  <title>Instruction test: snippet and insert</title>
 </head>
 <body>
  <opt:insert snippet="header">
   <h1>I'm a standard header</h1>
   <p>Foo bar joe</p>  
  </opt:insert>
  
  <hr/>
  
  <opt:insert snippet="content">
  	<p>Well, i'm also a standard content.</p>
  
  </opt:insert>
  
  <hr/>
  
  <opt:insert snippet="footer">
  	<p>And I'm a footer.</p>  
  </opt:insert>
  
  <p>&copy; Pasteright 2008 by LMAO, It seems to work!</p>
 </body>
</html>

The extending template (test_inherit_1.tpl):

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<opt:extend file="test_inherited_a.tpl">
	<opt:snippet name="header">
		<h1>Webmaster Of Puppets</h1>
	</opt:snippet>

	<opt:snippet name="content">
		<p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Phasellus ut tellus id nulla adipiscing eleifend. Sed dictum accumsan ante. Nullam at nisl vitae elit aliquet fringilla. Praesent egestas eros eget tellus. Praesent id odio a sapien rhoncus vehicula. Nunc fringilla, diam eget euismod tempor, tortor metus tincidunt sapien, eu cursus magna tellus at risus. Praesent non tellus eget magna facilisis pulvinar. Praesent libero mi, adipiscing a, pharetra eget, condimentum sodales, mi. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos hymenaeos. Donec ac elit. Duis iaculis tortor a metus. Aliquam id purus et eros faucibus fringilla. Praesent quis quam. In lectus urna, fringilla sit amet, iaculis eget, aliquet ac, quam. Donec vulputate dui sit amet lectus. Aenean tempor, orci at pretium ornare, tortor tortor venenatis ligula, eget blandit nisi risus eget dolor. Duis nunc neque, sodales porta, viverra non, tristique eu, sem. Curabitur magna neque, blandit ullamcorper, congue quis, tristique ut, felis.</p>
	</opt:snippet>

	<opt:snippet name="footer">
		<p>Bye!!!</p>
	</opt:snippet>
</opt:extend>

Some PHP code:

<?php

	define('OPT_DIR', '../lib/');
	require(OPT_DIR.'opt.class.php');

	try
	{
		$tpl = new optClass;
		$tpl -> sourceDir = './templates/';
		$tpl -> compileDir = './templates_c/';
		$tpl -> stripWhitespaces = false;
		$tpl -> printComments = false;
		$tpl -> setup();
		
		$tpl -> parse('test_inherit_1.tpl');
	}
	catch(optException $e)
	{
		optErrorHandler($e);
	}
?>

And the result can be admired below:

Some facts:

If we modify any of the templates in the inheritance chain, the parser will notice it and recompile proper files.
If we remove any of "opt:snippet" instructions, the default content of "opt:insert" will be displayed.
There is a possibility to overwrite an existing snippet by the more important template. You will be also able to call the parent content, however it is not implemented yet (some kind of "opt:parent" instruction).
It can be mixed with "opt:include" and "opt:sequence" (the sequence system briefly mentioned in the introduction).

What is more, it's quite possible that the template inheritance will work faster than traditional way, although it will consume more disk space. Simply, if we have an inheritance like this: "A extends B extends C", the compiler puts all their code in the file belonging to the "A" template. It reduces the amount of disk operations (the execution of one bigger template is faster than the same content in five files). On the other hand, by default OPT still checks the file modification times, but fortunately, the number of tests is also smaller. The exact results will be available, when some benchmarks are done. Before we end, I'll say that two days ago I made a small test (a simple list with 15 items). The result:

OPT 1.x: 1850 req/s
OPT 2.0: 2100 req/s

Well, it looks promising, but we have to remember that the new opt.class.php file still does not have half of the options, which obviously will increase its size. Moreover, some optimizations are not implemented yet. But in fact, you can assume that the new features should not destroy the performance, and there is a chance that it will be even better.