Reasons for the changes

From the technical point of view, OPT is a pioneer project. Many features were never implemented in any existing template engine and at the first time it was sometimes hard to say what potential they actually offer. We were learning the library during the development process. Unfortunately, the improvements could not be invented and applied forever. One day we had to stop and say: `OK, now it is the time to make it stable'. The result is quite satisfactory, but the list of the things that could be done better was getting longer and longer. Some of the features were suggested by the users, the rest comes from us. OPT 2.1 is going to be the next step in the template engine evolution.

Compiler changes

In OPT 2.1, the compiler becomes a universal framework for implementing template languages, primarily based on XML. It provides the compilation process structure, interfaces and ports for many features, such as data formats. However, the details of the syntax processing are moved to external parts:

  1. Custom parsers - their task is to parse the template source code and construct an Abstract Syntax Tree (AST).
  2. Custom AST nodes - currently, the AST nodes match the specific OPT-XML needs and it is impossible to write new ones without changing the compiler code. In OPT 2.1 it becomes possible.
  3. Custom expression parsers - they parse expressions such as $a+$b. Since OPT 2.1 it will be possible to write new ones.

In other words, OPT is going to bring the compilation structure, whereas the programmer matches it to his or her needs. Personally, I hope that someone will use it and port Template Attribute Language to OPT. However, most programmers will not be interested in the details of compilation process, but rather the default language it offers. Here, OPT 2.1 language will be very similar to the current language. It will be just ported to the new API.

New parsers

By default, OPT 2.1 will offer three parsers that could be used by the programmers. They will replace the current `compiler modes':

  1. XML - completely new parser based on the XMLReader extension. Its primary feature is 100% sure full compatibility with XML standards, including such details, as xmlns attribute. However, the parser cannot be scaled to ignore some syntax requirements.
  2. HTML - the current parser, customizable with many options.
  3. Quirks - the quirks mode parser.

New dynamic attribute value syntax

As OPT is based on XML, there occurs a problem, how to put an expression value into an attribute. In the early development stage, we decided to make use of namespaces to indicate the attributes which have dynamic values. The life shown that it was not the best idea. It is not scalable and causes problems, if the attribute already contains a namespace. Redesigning this feature from scratch was one of our priorities. In OPT 2.1, the value type will be encoded in the attribute value, not in the name. It is illustrated by the following examples:

<p class="text">Static attribute value: text</p>
<p class="parse:$text">Dynamic attribute value read from a variable</p>
<p class="null:parse:tekst">Static attribute value: parse:text</p>

It can be also used to select the custom expression syntax:

<p class="foo:##mysteriousSyntax">text</p>

And in curly brackets, too:

<p>Mysterious syntax: {foo:##mysteriousSyntax}</p>
<p>A bit nonsense example ("parse" is the default): {parse:$variable}</p>
<p>Even more nonsense example, but still possible: {str:$variable}</p>

And in instruction attributes:

<opt:include file="parse:$file" />
<opt:attribute name="str:theName" value="str:value" />

Extended expression modifiers

In OPT 2.0, there were two hardcoded modifiers used for the HTML escaping control: e: and u: used as follows: {e:$expression}. Since OPT 2.1, it will be possible to create custom modifiers and use them in the attributes. Furthermore, there will be three predefined modifiers that could be overwritten:

  1. To enable the escaping (by default e:)
  2. To disable the escaping (by default u:)
  3. To enable the escaping in attributes (by default a: - the new default modifier)

The reason to introduce the new default modifier were the needs of HTML escaping and XSS protection for attribute values that must be a bit more complex than for normal tag body. You will be allowed to use the more complex escaping function for attributes, and the simpler, but faster - for ordinary texts.

Some of you may have noticed that the modifiers use the same syntax, as the expression type chooser. Nothing more further from the truth! Thanks to some simple conventions everything is unambiguous. First of all, to specify both the expression type and the modifier, we specify the parser, and later the modifier: {parse:e:$variable}. Secondly, the modifier name is always one character long. At at last, in the code like this: {e:$variable}, the compiler matches the modifier first and then the expression parser.

Extended data format support

Data formats are probably the most brilliant feature of OPT and in the next branch this element will be greatly improved. Let's take a look at instructions first:

<opt:section name="foo">
 ...
</opt:section>

To specify the data format for thi section, we have to link it with a variable (in this case: $foo), but it is not always possible. For example, the variable may be specified by the datasource attribute or the section could have nothing to do with variables. Such situation occured while developing Open Power Forms. Some sections must be connected to the form and have no equivalent in template variables. And what if we would like to write a new implementation of opt:include? In OPT 2.0 it is impossible to pass the information about the data format to it.

The problem will be solved by introducing a special, optional attribute called id. Obviously, it will allow to define an unique identifier for the instruction which can be used to find the data format definition or modify any other aspect of the behaviour. It leads to the extension of the list of the available data formats. Some of the new ones are:

  1. Include - the default data format for opt:include
  2. System - the $system special variable handling, moved from the compiler code to the data format.
  3. Scalar - the scalar data (text, numbers, logical values) that does not allow the container, array and object access. It will be impossible to use them as section data sources which will improve the template code security and error resistance.
  4. RuntimeIterator - if you do not want to write a custom data format and Objective does not meet your requirements, but you still want to use sections with your own objects and their own logic, this format will allow this. It will work with a special interface that needs to be implemented in your class that provides the whole section functionality.

Snippet arguments

Snippets resemble macros from such languages, as C. A new interesting addition in OPT 2.1 will be the possibility of giving a snippet the arguments. I have not invented a syntax for this element yet, so please treat the following code as the illustration of an idea:

<opt:snippet name="snippet">
 <p>{$info.foo}</p>
 <p>{$info.bar}</p>
</opt:snippet>
 
<opt:section name="foo">
<div>
 <p>Some content</p>
 <opt:insert snippet="snippet" info="$foo" />
</div>

Now the snippet variable $info will be treated as $foo, keeping its data format and features. It will make the snippets much more portable.

Snippet loading

Currently, the shared snippets can be included in our template in two ways:

  1. Template inheritance
  2. opt:root attribute include

The last way will be extended by a special tag opt:load which will allow to load multiple templates with snippets:

<opt:root>
  <opt:load template="snippets_1.tpl" />
  <opt:load template="snippets_2.tpl" />

  <!-- the code here -->
</opt:root>

Template functions

Snippets have one important disadvantage: they are static and exist only during the compilation. It is impossible to select the snippet name from a variable or something like that. Many people ask for such a feature, and the problem became known as "dynamic snippets". I did a long research looking for the possible ways of adding this feature, but due to the PHP language construction and the snippet features, it seems to be impossible. The snippets cannot be implemented as functions or class methods, because:

  1. The access to the template variables {@variable} will be lost.
  2. The access to some internal compilation code variables will be lost, which could lead to serious problems with data formats.
  3. The snippets would be strictly connected with one, particular data format, whereas they should match the current data format in the place of insertion.

This is why I decided to introduce a new language element which does not have an official name yet, and during the development process we call it "template functions". Why? Because it is a reimplementation of the classic PHP function features. They are created once and physically called, intead of being copy-pasted during the compilation, like in snippets. What is more, they will support a dynamic selection of the called function.

My plan is to implement them so that they could be used in many places, where the snippets can be currently applied:

  1. Section bodies
  2. Components
  3. Blocks

Of course some of the locations will be unavailable for them and moreover, they will be strictly connected to a single data format. An interesting feature is that they could be injected into the component objects, like (please remember that the syntax is not invented yet):

<!-- define a layout of the checkbox list for the checkbox component -->
<opt:function name="checkboxListLayout" list="required">
  <ul>
  <opt:section name="list" datasource="$list">
    <li>{u:$list.checkboxCode}</li>
  </opt:section>
  </ul>
</opt:function>

<form:checkbox-list name="list">
  <label for="parse:$system.component.id">Select something:</label>

  <!-- dynamically select the layout to inject -->
  <opt:display inject="$system.component.layout" />
</form>

Improved section syntax

Together with the current syntax with opt:show and the empty section tag, in OPT 2.1 it will be possible to use the new style presented by the example below:

<opt:section name="foo">
	<opt:body>
		<ol>
			<opt:item>
			<li>Dodood</li>
			</opt:item>
		</ol>
	</opt:body>
	<opt:else>
		<p>Hi universe!</p>
	</opt:else>
</opt:section>

As we can see, in place of opt:show we write the section tag, and in the body we use opt:body to indicate the section body and opt:item to specify a place which will be iterated. The current syntax will not be removed, as they are complementary to each other, mostly when it comes to the cooperation with snippets. I was also asked for the possibility of having the section body enclosed in an extra tag which is quite useful if we have an extra opt:else at the same place, and want the body to be loaded from a snippet.

Moreover, the small changes will be applied to the opt:selector instruction:

<opt:selector name="foo">
	<opt:select item="fff">ssdf</opt:select>
	<opt:select item="ssddf">ssdf</opt:select>
	<opt:select item="abc">ssdf</opt:select>
</opt:selector>

In other words, it will be no longer necessary to create thousands of new tags for our choices.

Switch instruction

OPT 2.1 will introduce the new instruction: opt:switch. Contrary to the switch statements from the classic programming languages, it will be a real swiss knife. Take a look:

<!-- classic switch -->
<opt:switch test="$condition">
 <opt:equals value="value1">Body...</opt:equals>
 <opt:equals value="value2">Body...</opt:equals>
 <opt:equals value="value3">Body...</opt:equals>
 
 <!-- an equivalent of playing with "break" -->
 <opt:equals value="value4">
   Some content...
   <opt:equals value="value5">
     Some content...
   </opt:equals>
 </opt:equals>
 <opt:default>...</opt:default>
</opt:switch>

If you like the power of case... break construct in PHP, you will surely enjoy this stuff:

 <opt:equals value="value4">
   Some content...
   <opt:equals value="value5">
     Some content...
   </opt:equals>
   Some content...
   <opt:equals value="value6">
     More optional content
   </opt:equals>
   The ending
 </opt:equals>

If the tested condition equals value5, we will see just the contents of the value5 tag. But if it equals value4, we will see its contents, together with value5 and value6. But this is not the end. OPT switch will work with... containers:

<opt:switch test="$container">
  <opt:contains value="foo">body...</opt:contans>
  <opt:contains value="bar">body...</opt:contans>
  <opt:contains value="joe">body...</opt:contans>
</opt:switch>

In this case, OPT will be able to display the values of more than one choice. opt:contains could also be nested, similarly to opt:equals. And what would you say, if you see that they can be mixed?

<opt:switch test="$condition">
  <opt:contains value="foo">body...</opt:contans>
  <opt:contains value="bar">body...</opt:contans>
  <opt:contains value="joe">body...</opt:contans>
  <opt:equals value="value1">Body...</opt:equals>
  <opt:equals value="value2">Body...</opt:equals>
</opt:switch>

In this case, OPT simply checks the expression type. If it is a container, it matches the opt:contains tags, otherwise - opt:equals. Nested cross-mixing of opt:equals and opt:contains is also possible.

We get even better functionality, if we use the attribute form of opt:switch:

<ul opt:switch="$row">
 <li>Name {$row.name}</li>
 <li>Age: {$row.age}</li>
 <li opt:contains="city">City: {$row.city}</li>
 <li opt:contains="email">E-mail: {$row.email}</li>
 <li>Date: {$row.date}</li>
 <li opt:contains="other">Other: {$row.other}</li>
</ul>

And the finale that will help in the last example. Suppose that we would like to add the class="last" attribute to the last displayed list element. This could be done by the following code:

<ul opt:switch="$row">
 <li>Name {$row.name}</li>
 <li>Age: {$row.age}</li>
 <li opt:contains="city">City: {$row.city}</li>
 <li opt:contains="email">E-mail: {$row.email}</li>
 <li>Date: {$row.date}</li>
 <li opt:contains="other">Other: {$row.other}</li>
 <opt:append to="last">
   <opt:attribute name="str:class" value="str:last" />
 </opt:append>
</ul>

For everyone who claim that "PHP is the best template language": enjoy implementing similar functionality manually.

Futher changes in syntax

During the development process, there have occured some inconsistences in the naming style of the instructions. OPT 2.1 will attempt to fix them. Basically, all the tags named opt:someTag will be renamed to the form opt:some-tag. In sections, monsters like opt:sectionelse will be shortened to opt:else. opt:use will be renamed to opt:insert, and opt:on into opt:omit-tag.

Inflectors

This feature concerns the public library API and is related to finding the templates in the filesystem. I was talking to some programmers and the developers of CMS-es etc. They were often complaining about the current, hard-coded algorithm of mapping the template names to the files based on the streams. They said that it is very impractical, when it comes to the modular applications - it is very hard to manage the registration of dozens of quasi-streams, and operating them on the script- and template side. They would enjoy a system that would give them more flexibility. This is how the idea of inflectors was born. An inflector is a simple class that gets the "public template name", such as `db:user/edit.tpl` and returns the real filesystem path to it. The default OPT inflector is basically the reimplementation of the current template locating algorithm. It is just extended by an extra API for managing the streams. However, if the programmer needs some specific rules, he can write a custom inflector and deal with it on his own.

Backward compatibility

OPT 2.1 is intended to be used in new projects, however - if you stared writing something with OPT 2.0, you do not have to worry. Although the syntax changes can bother many programmers, there is no need to worry. OPT 2.1 will have a backward compatibility mode. Basically it activates the whole old and replaced syntax, with keeping the new features at the same time. You can use this mode to make use of the new features and give yourself some time to make a migration. What is more, many changes are just syntactic, not semantic. I plan to write a special tool that would convert the existing templates to the new syntax with one click.

Note that if you actually want to be up-to-date, the migration at some level will be necessary. The OPT 2.2 branch will be basically OPT 2.1 with the backward compatibility code removed.

Support

If you want to stay with OPT 2.0 in your current projects, the support for this branch will be continued long after the OPT 2.1 will be released. We will fix the encountered bugs and continue improving the stability of this branch. Actually, OPT 2.1 is going to be a transition branch to the next evolution level. The plans are as follows:

  1. OPT 2.0 - works on PHP 5.2, the support is continued for at least one year after releasing OPT 2.1
  2. OPT 2.1 - the new branch with new features and the full backward compatibility mode. The minimum requirement is PHP 5.3
  3. OPT 2.2 - backward compatible with OPT 2.1, but without the 2.0 compatibility mode. Also, the very long support is planned.
  4. OPT 3.0 - basically the same as OPT 2.2, but with the entire code migrated to PHP 6 and namespaces.

Note that OPT 2.0, 2.1 and 2.2 are also supposed to work on PHP 6 as soon as it will become more stable, because nowadays I even have serious problem with compiling the development snapshots.

Changes to the OPL core

As you can see, the new branches will require the improvements of the OPL core. The planned features are:

  1. OPL 2.1 - new features: namespace support, command line interface tools, new debug console implementaton, improved error handling.
  2. OPL 2.2 - removing legacy code for compatibility with OPL 2.0
  3. OPL 3.0 - migration to PHP 6 and namespaces

Conclusion

OPL project is getting more and more mature. The new features will make it more suitable to deal with new problems and situations, where the template engine support is needed. At the same time, I think about how to introduce the new users in the project and teach them, how to use the offered tools. You can expect new tutorials, and personally, I think it would be nice to write a book after releasing OPF and OPT 2.2...