Right, I am just about to check in some new code that makes will make it very easy to import/prepare Creole for the ZenMagick loader architecture. The new (mostly unused) class ZMPhpCompressor also allows to squezze a full Creole release (everything under classes) into a single 188kb file.

I am actually not sure if I should include that already with the next release, as right now there are probably about 3 or four queries per request done by the new API and I might as well stick with the existing code a bit longer - we’ll see.

The code depends on two other ZenMagick classes, so it’s not very portable (if anyone is interested), but apart from that it works fine

The actual code to do the magic looks like this on my XP box:

$creole = 'creole-1.1.0';
zm_creole_import('C:/TEMP/'.$creole);

$comp = new ZMPhpCompressor();
$comp->setRoot(’C:\Program Files\Apache Group\Apache2\htdocs\zen-cart\zenmagick\core\ext/’.$creole);
$comp->setOut(’C:\Program Files\Apache Group\Apache2\htdocs\zen-cart\zenmagick\core\ext/’.$creole.’.php’);
$comp->setTemp(’C:\Program Files\Apache Group\Apache2\htdocs\zen-cart\tmp’);
$comp->compress();

Not sure how often I’ll be using this, but it’s nice not having to try to remember these things once they are needed again!

The only caveeat is that compressing into a single file breaks the Creole ‘dot-path notation’, so in order to use the MySQL driver you’ll need to do something like:

Creole::registerDriver('mysql', 'MySQLConnection');

to avoid errors about missing drivers. Since all database access in ZenMagick is handled by a single database provider class that’s not really an issue, though.

That means the last open issue is to write some proper release notes - oh, well, babysteps

I’ve been trying to get a release together for quite some time now, but as laid out before, I was unable to resist to add this ‘last new feature’ before releasing.

Due to the massive amount of changes (even though old themes will be good with a handful of small changes) I think I’ll actually have a release candidate before doing a proper one.

There are currently three remaining issues, though:

• Creole
  The script to “import” the Creole code (read: fiddle around to make it work with ZMLoader) is still custom and ideally I’d prefer to have a single compressed creole.php file rather than the single files. This will require splitting the core compressor code into a generic bit and the ZenMagick specific parts. Once that is done I’ll be able to compress individual folder and subtrees just as needed.
• Release notes
  My old nemesis. With the growing number of changes it’s becoming imperative to have some good release notes so people can make an informed decision about whether they want to upgrade or not.
  Also, it should contain enough details to make theme migration an easy task
• Time
  As some of you might have noticed I am not online a lot at the moment. There are some good, personal reasons for that so expect this continue for a while.

If anyone is interested to give the current code a try let me know and I’d be happy to put a custom build together for you (there is also the option of accessing the trunk code directly at sourceforge.)

I feel it is about time for a new release. I have been quite busy and actually I am quite pleased with the results so far.

The work on this release has not been very structured - I think I should perhaps start making plans for new releases and at least try to stick to that.

On the other hand, there have been so many changes and dependencies that it would have been very hard to follow that plan for 0.9.1 to be.

I am now in the final stage of actual changes - thanks to Matt who opened a whooping 21 tracker in on session this has proven to be a bit longer than expected It’s great to get feedback and there are a few really good observasions in those bug reports and feature requests.

The last thing to be added is something called toolbox. The original bug that started this is about inconsistencies in when and how text is HTML encoded/escaped. Another issue that came up a few times lately is the fact that it is not very easy to customize the code in the html and html.defaults packages.

To solve both of these issues there is now the new toolbox. Basically, it allows to register objects (via local.php or similar) that then will be available for all templating code to use.

Most code in core that generates HTML or any other output that is used in templates is now available in two different ways:

1. There is a new variable $_t that can be used to access all registered tools.
2. Tools will also be available via their name (see examples further down)

The toolbox contains 4 build in tools that are going to replace the current set of html (and related) functions:

• html - Everything that actually creates not just text, but actual HTML (single tags only)
• net - All href helper
• form - HTML Form specific tools
• macro - Container for all functions that are currently under html.defaults

The advantage of using classes is that now it is possible to replace existing methods rather than having to copy and rename functions.

For example, rather than writing:

<?php zm_secure_href(FILENAME_LOGIN); ?>

one can use now:

<?php $_t->net->url(FILENAME_LOGIN, '', true) ?>

or:

<?php $net->url(FILENAME_LOGIN, '', true) ?>

Explanation: I have merged zm_href and zm_secure_href into a single url(..) method (same for zm_form and zm_secure_form). That explains the additional two parameter (params and secure/unsecure flag).

It is also possible to register new tools by doing something like:

<?php

$mytool = new Foo(); // or ZMLoader::make(’Foo’);

ZMToolbox::instance()->set('mytool', $mytool );

?>

In templates the object $mytool is now available as $_t->mytool or directly as $mytool.

I also reviewed all converted code with an eye on HTML encoding and improving the documentation. My tests do not show neither loss nor gain in performance, so all should be good.

All existing functions have been marked as deprecated and are converted to using the toolbox. That means even when not upgrading an existing template to use the new variables, the new code will still be used.

Only thing left is to convert the default and demo theme to using the new code, but I think I can do some automated rewriting that should help a bit.

I am (again, again, again) agonizing over ZMCategories.

As part of the database code changes I am reviewing all services and model classes and there is heaps that could be improved (from a coding point of view, most of this work won’t be visible to regular users that just mess with templates).

There are actually two things that make this rather tricky:

• Internal caching
  The current implementation does a lot of internal caching in order to be fast.
• The active flag on categories (via applyPath(..))

Initially caching was not a big issue until I added support for multiple languages. Now that I want to add support to create and update categories (for admin tools) there is a second dimentions: the status. Further complications are that the code builds a tree structure to support the category tree navigation.

The second is actually just something I’d rather implement differently as it depends enirely on the cPath request parameter and that shouldn’t interact with a service.

So, I plan to change two things:

1. The meaning of the method isActive() in ZMCategory will change and be similar to the same method in ZMProduct.
   Active categories are visible in the store, inactive aren’t.
   This should actually not be a big issue as this method is used only in the default categroy rendering method zm_build_category_tree_list().
2. The whole logic currently done by ZMCategories::applyPath() will have to go somewhere else. Again, potentially the only code affected is zm_build_category_tree_list().

Any objections? - let me know…

EDIT: As it turns out,  changes are really confined to zm_build_category_tree_list(). That means unless your theme uses a custom version of zm_build_category_tree_list() there is nothing to fear

It’s been a while so I think it’s time to write up a few things that currently occupy some bits of my mind.
1) Donations
I promised not to write about this any more, so just a big Thank You to everyone that that showed appreciation of what I do

2) Making use of all the architecture work I put into ZenMagick
Today I had the chance to show off some of the framework features that make ZenMagick tick. Involved are:

• Event handler
• Custom validation rules
• Use of services to update account information
• Best of all: no changes to core files!

The problem was to:

1. Enforce collection of a phone number during guest checkout
2. Default the first/last name of the guest checkout to the shipping (or billing) address

If you are interested in the single file solution to this check out this thread on the ZenMagick forum.

3) The database work in progress

One sideeffect that I haven’t talked about is that the new code will force me to finally add all the missing setXXX() methods to model classes. Yay! Also, since the changes will make use of automatic table <-> model mappings we’ll get update functionallity for all sevices pretty much for free!

I hope that being able to create and update data in the database will make the API even more valuable (I know that there are a few people out there waiting for this…)

 4) Backwards compatibility

I have written about this before, so just to reinforce the message: 0.9.1 will be, using a single setting, mostly compatible with 0.9.0. Mostly, because there are a couple other changes (zm_product_href got another parameter, for example) that will require some minor changes. There are also a number of functions deprecated - I think I’ll add another setting to log/backtrace using those. That should make upgrading a lot easier.

5) Framework

There are actually still things that I won’t be able to do in a single release cycle. One thing I’ve been dreaming about would be to extract the actual core framework and make it a separate product(?).

The value of doing that would be to be able to concentrate entirely on making the framework as good as possible without the contrains of an actual application using it. I suppose that is something that could happen once ZenMagick hits the 1.0 version mark.

Writing a database abstraction layer is definitely not something I had in mind when stating ZenMagick On the other hand I really enjoy this sort of work - in fact much more than working on new store features.

So far I have a single interface ZMDatabase that will eventually be used to access the database. Right now there are two implementations and the implementation class is configured via a setting.

The interface is not complete yet, I’ll add stuff as a go through the code and convert database access. Currently there is only ZMManufacturers upgraded to use the new code and it works perfect.

Only worry right now is that Creole may throw Exceptions. In itself that is not a problem - is’s actually a nice change to know whether things have worked or not. However, currently none of the code is the least exception aware.  I probably should change that before going to far down that way to avoid having to redo a lot of changes.

I just fear that by starting that can of worms I’ll end up rewriting most of the current dispatch code in order to properly handle exceptions throughout the request process… Oh, well, one step at a time…

Today I spend some time massaging Creole to properly load in ZenMagick using either ZMLoader or as part of core.php.

There were a number of issues I ran into:

1. include and require statements in Creole sources rely on the PHP include_path
2. Some files are missing the closing ‘?>’
3. DebugConnection.php does not include/require Connection.php, breaking the otherwise perfect dependencies
4. Record.php includes a require for QueryDataSet.php which actually creates some sort of circular dependency loop, since QueryDataSet.php also depends on Record.php…

#1 was to be expected in one form or another, so it wasn’t really an issue.
#2 is actually something I do not really understand, but then, I never had the time/energy to read the PHP specs to figure out why this is working at all. If you know, feel free to leave a comment…

#3 turned out to be a problem, because in order to be able to include all files in core.php I had to rearrange all source files based on dependency. A short script that build a simple dependency tree was easily done based on include/require statements, except for this one file (which I ended up hardcoding in the script…)

#4 was really annoying because it meant that initially I had to patch Record.php in order to get all files loaded at all using the include_path solution (which is the documented way of installing Creole!) . Eventually I changed my *import* script to load the files from a non ZenMagick folder, so no initial loading was done.

One thing that might come out of this exercise is that hopefully I will have time to split ZMCoreCompressor into a generic class and a ZenMagick specific subclass. That should allow me to easily build a single file Creole version that can then be included in ZenMagick versions instead of the full 90 odd single files (this is version 1.1). I might strip this down, though, excluding all non MySql drivers as there is currently no point loading them…

One really nice thing I discovered was that Creole follows a concept of filename-classname relationship very similar to ZenMagick. That definitely made it a lot easier!

As a first step of migrating to a more sophisticated database access system I have started with adding a simple layer (read: interface) that eventually will replace all $db usage. The interface will provide some simple methods to read/write the database using a lot of the ZenMagick binding code.

That should make it easy to replace the current $db based implementation with something different. The nice thing right now is that I can migrate services and other database access classes one by one without breaking anything.

Once all code is converted to using the new interface I’ll start investigating alternative implementations. I really like Creole, as it looks (intentinally) a lot like Java and the interface design is done with that in mind, so it will be easy enough to create an implementation using Creole…

I have just checked in some new code that changes the way settings are accessed. While doing that I went through the current set of default settings and started cleaning up.

There are quite a number of unused settings in there right now, so that has to change.

Another thing I realized is that there are actually a number of definitely layout settings. Those are still from the old days and actually they do not fit the ZenMagick philosophy any more.
I think most of them will have to go. I will try to add a simple way to ‘have them back’ for portability. The default theme will change to live without them. Templates will be changed to look like the current templates using the default settings.

As a side note: it is always possible to use the zen-cart defines in ZenMagick. It is not required to use settings. So, it should be easy enough to replace somwething like the current zm_setting('isSiteMapAccountLinks') calls with SHOW_ACCOUNT_LINKS_ON_SITE_MAP=='Yes'

I’ve been pondering a few database related issues lately. There are two main things that bug me right now:

• I am not too happy with the current database code in zen-cart
• The ZenMagick database code is getting at it’s limit right now

In the long term I would like the database code to move towards an existing db layer, for example creole. That would be still pretty low level, but considering the complexity of the current table layout I would prefer something simple and build custom code on top of that.

My vision for the ZenMagick SQL code is to convert that incrementally to:

• Using an existing basic db access framework
• write something smart and custom to generate the SQL with less custom code for each service

I think the second point is actually the more crucial and interesting one

One first step that I have started already would be to clean up the model classes. All methods with actual logic should move somewhere else. Ideally, there would be only get/set methods and even that could be reduced.

The main problem I see right now is that by removing the actual getXXX() and setXXX() methods the phpdoc documentation would suffer. Right now I am undecided what to do. It would be nice to have just a simple model class with 10 lines, no matter how many properties. On the other hand it would mean people would have to start looking at the database tables again in order to find out what a model class actually contains!

One solution would be more code generation based on some meta data. However, that would be something happening during release build, not at the users side (like the core.php stuff).

Any opinions welcome. Some educated guesses as to what zen-cart is up to would be very helpful too!