Some time ago I wrote about some changes to make large catalogs more performant. Today a second batch of changes got committed (sorry, forgot to push, though!).

What does that mean?

First of all, this was in collaboration with a developer facing issues with a large database, in particular categories. So, talking to me does occasionally help getting your problems solved faster!

Secondly, there is now a new interface ZMSavantCache and a also a simple implementation class thereof. There is also sample code of how to configure Savant to use the cache class and how to configure the cache class itself.

All there is to do is:

1. Change the bean definition of the default view class to set the caching implementation to be used
2. Set up a new setting that lists all the template filenames (for example: ‘boxes/categories.php’) that can be cached
3. Done!

Eventually this should be the default setup with some reasonable defaults (perhaps with a more sophisticated cache implementation, though).

This template file level caching is a great way to complement the existing page cache plugin. In the above example of a catalog with thousands of categories, caching the categories box cut of about 4 seconds of processing (measured inside a VM, to be fair).

In addition I finally managed to get APC working on my dev environment (Windows, go figure…), so there is also a shiny new APC cache implementation to go with that

And, again, all that just because someone started asking questions and suggested solutions and patches.

You are welcome.

With the ongoing progress in the new admin UI I’d like to step back for a moment and review the current template organization.

So far, the templates use a similar approach as the storefront code. Meaning there is a central layout file that controls the overall page structure. The actual variable page content is rendered by an individual view template for each request.

There are some obvious advantages in this system:

• Central layout
  Changes to the general page structure are done in a single place
• Simple views
  All a view has to do is to create HTML for a rectangular box – the view. That content then gets stuffed into the right place by the layout template – done.

However, there are also downsides:

• No control over anything outside the actual view
  Beside the actual view contents there are a number of other elements on a HTML page that are also request specific. Typical examples are: page title, meta tags, crumbtrail, custom CSS and JavaScript.

I am sure there are more, but the above is what I’d like to look at in this post. While there are reasonably well working tools to handle custom CSS and JavaScript, there isn’t really a great solution for some of the other elements.
Of course, I could write even more tools classes to manage each and every element outside the view, although that looks like a lost case as there will always be new things to consider.

So, what’s the alternative?

The other extreme, of course, would be to code the full page structure on each view (and drop the layout, which can be done). It is hopefully clear that this can’t be the answer. I still have to see a project where this approach, over time, hasn’t lead to diverging page layouts between pages.

So, the question is: is there a middle way?

One thing that would become clear quite easily when trying to code each page in full is that common elements would be extracted and stored in separate template files for all to (re-)use. An technical alternative would be to wrap common content in functions (or macros, depending on the template engine).
This would allow each page access to all relevant page elements while minimizing the duplication of code/markup [basically reduced to html, header and body tags).

Perhaps something worth experimenting with before committing any further to the current system. I am willing to spend some time to investigate this in practice before the work involved to change all templates becomes too big.

It’s certainly going to be interesting and I will post more about it as I go.

I just realized that the wiki still claims that there aren’t any themes available for ZenMagick. That is, of course, not true!

I’ve updated the wiki’s theme repository page with some information and links to the demo store for each theme. Seing that themes/templates are a popular search topic, I hope that this helps spread the word…

Last night ZenMagick 0.9.5 was finally released! I am quite excited about this particular release, because it includes a couple things that make ZenMagick a lot more attractive to existing Zen Cart installations.

For one thing, there is a new patch that allows plugins to access the final HTML content of Zen Cart templates. This has been around in ZenMagick (for themes) for quite a while and proves to be very powerful.

A lot of the storefront plugins (page stats, page caching, google analytics) depend on this feature and now they can be used with Zen Cart templates too!
If you head over to the demo store you’ll see that the HTML generated by the default Zen Cart template is caches like any other page, following the same rules!

Another quite surprising thing (at least  for me!) is the new PDO based ZMDatabase implementation. It seems to be the fastest implementation so far and the demo store seems to do fine using it, so I might make it the default implementation for the next release.

There are also improvements to existing plugins and, of course, a couple new ones. The most interesting one is probably the phpBB3 plugin. It provides the same features as Zen Cart’s phpBB support, just for phpBB3.

I also really like the new theme toggle plugin that allows to switch between Zen Cart and ZenMagick. Probably not very useful for a production environment, but good to compare features, displayed values, etc.

Of course, this is not the end, and I’ve already started working on the next version. In that context a reminder that with 0.9.6 I’ll be removing a lot of the deprecated stuff. Also, the deprecated API support will be removed before a 1.0.0 release, but most likely not before 0.9.8.

I’ve just updated the demo store with a release candidate version of ZenMagick 0.9.5.

There are some interesting visual and other changes that the demo store illustrates:

• Use of PDO based database API
• New plugin to switch between ZenMagick and Zen Cart template
• Improved event support for Zen Cart templates (that’s what the switch is really for!)
  The extended event support now allows ZenMagick plugins to manipulate the final HTML of Zen Cart templates. That means, all ZenMagick plugins that rely on this mechanism can now be used in Zen Cart templates, too! Obvious examples are the page stats and style switch plugin (even though the style switch doesn’t really work for Zen Cart templates, right now).

One thing that is now good to see is the difference in database queries. Even though the Zen Cart default template and the ZenMagick themes do not display exactly the same data, it’s still impressive to see the difference in queries

I somehow lost my release notes so the public release will have to wait until early next week to give me time to get organized again.

Happy demo’ing!

I happened to notice a  search ‘when is zencart 2.0 actually coming‘ in today’s log. It’s a good question. Unfortunately I do not know the answer, but I do know that the next ZenMagick release is not far!

Today I checked in the final changes to make the category pages use the new sql based pagination code that hopefully will be on par with the way Zen Cart handles it [(except for being a lot more complex, of course ].

That means that the last bit I was hoping to get into the next release is actually in place. I’ll be doing some more testing and try to figure out which plugins need updates. That should hopefully get me to a point where I can write release notes and build ZenMagick 0.9.5 next weekend!

I think I really need to do something about the plugins. Ideally, they would all have their individual changelogs, but right now with core changes that affect pretty much all plugins that seems to be overkill. I guess for 1.0 that might be something to look into, though.

Coming back to the search mentioned above: Of course there is no need to wait for Zen Cart 2.0 in order to enjoy things like:

• Plugins that do not require file changes
• A simple template structure that does not need include/require
• A database API incl. DAO‘s, transaction control and support for different database providers
• A MVC based framework
• Flexible form validation with rules to change both JavaScript and server side validation in a single place
• Guest checkout aka. checkout without registering
• and a lot more

Just start using ZenMagick today!

With the number of things on my todo list for the next release shringking fast its time to have a look at what is about to happen in the next ZenMagick release.
I’ve already written about a few things, but I think it is also nice to have it all in one place. (Also, these posts are usually good starting points for writing the release notes!)

So, here’s a list of things to look out for int he next release:

• Banner Management
  So far, banners where identified by index (integer) and needed a setting each in order to work. As of 0.9.5 this is going to be simplified and there is actually already some documentation about how things are going to work.
• Database API
  A number of things changed here, in particular the ability to create type mappings on-the-fly and support for array model data in all xxxModel() methods should make life simpler in a few cases
• define refactoring
  A lot more defines have been converted into consts in the appropriate classes. Defines are fine, however, I feel that consts are a lot more readable and also it’s easier to find where things actually are in the source code.
• ZMLayout refactoring
  The service class ZMLayout has been renamed ZMTemplateManager in reflect the newly added methods to manage CSS and JavaScript resources. Short story is that it allows to add CSS to a page even after the <head> section has been created by the template.
• Support for large result lists
  I’ve explained this already, so I’ll just say that the new solution is SQL based and pretty close (at the bottom) to what Zen Cart currently does.
• Plugins
  As usual there will be new plugins. First, an initial version of a plugin to manage custom ZenMagick settings in the database. It uses some new widgets and I expect that with the number of widgets growing there will be a lot of improvements in the future.
  Secondly, support for phpBB3 at the same level as currently exists for phpBB2.
  There are also improvements and fixes for a number of other plugins that are, of course, listed in the changelog.
• Open
  A couple things are still open, but should be included, time permitting: a new search page based on the already existing ZMProductFinder class, make all unit tests pass!

Last week I was asked if it would be possible to use the CSS click-show-hide menu mod in ZenMagick. After some quick testing it turned out that only some minor changes are required in order to get it working.

Of course, there are some things that could be nicer. With product counts enabled the generated category tree uses an additionl 260 database queries (even thought that is not really news).

I changed the mod to using the ZenMagick API which kind of fixed that problem. One minor issue remained, though. The original Zen Cart code calcualtes the product count on a category sub-tree, not just the actual category. So the numbers are not quite the same. I might fix this, even though I doubt that this is a major showstopper.

The really interesting question now is how to wrap up the resulting files. All there is are the original javaSript, CSS and image files (CSS image path adjused to suit the ZenMagick file structure), the class that generates the HTML and the sidebox template file.

(more…)

Thinking about events yesterday made me wonder why there is actually a very specialised method filterResponse() for plugnis. One of the reasons was that I thought it would be nice for regular code to be able to modify the response, too.

One example of why that could be useful is CSS. In contrast to importing JavaScript, CSS needs to be imported/references in the <head> section of a HTML page. That means if the code that generates the HTML needs some special CSS this needs to be handled way in advance of where the code is actually executed.

One idea is to have some sort of resource manager that is able to access the final HTML and inject all the CSS [either explicitely or by linking .css file(s)]. That way the fact that new CSS rules are required would not need to be known in advance.

One obvious use case would be the new widget class, of course!

Looking at the code I remembered the real reason why this is so special: the HTML contents is passed into the method as parameter and the modified contets returned used as parameter for the next plugin, and so on.

Currently, events do not return values. The only parameter is a hash map that may contain any type of data.

So, to cut to the point, I changed the fireEvent() method to optionally handle return codes. The default behaviour is compatible to the current use, however it is possible for event subscriber to return a changed array.
Strictly speaking this is not really necessary as it is possible to modify the parameter array due to the reference handling in PHP5. Still, should special events require a contract where the parameter is not an array things should be ok.

So, as o f the next release [0.9.5] content filtering will be available to every piece of code that whishes to do so.

Zen Cart does have an extensive event system. One thing I feel make it not very useful is that most events are specific to a particular page. For example there is a different NOTIFY_HEADER_START_XXX event for each different page.

In ZenMagick there is the generic controller_process_start event with the actual controller instance as event parameter. In particular plugins that need to run on each page this needs some additional work. (Not much, but there are worse examples).
So, one other outcome of my work to make plugins work (better) with Zen Cart templates is to convert Zen Cart events into ZenMagick events to tigger the same event callbacks from within a Zen Cart template.

Seeing the events using the page_stats plugin is acutally quite impressive. For example a product details page (demo DVD product) looks something like this in Zen Cart:

Compare that with the same page using a ZenMagick theme:

Again, both pages where rendered without any sideboxes.

You can spot the artificial ZenMagick events in the Zen Cart event list as the keys are lower case. The differences are even more impressive with products that have lots of attributes.