Archive

Archive for August, 2009

Performance Optimized httpd.conf for Magento eCommerce

August 31st, 2009
Comments Off

If you ever run Magento eCommerce on your servers you know that it requires some serious horse power. It is just the nature of the beast, with XML configuration, the usage of the Zend Framework, and MVC design – every page load is not trivial. I realize that there are many lighter and various other forms of web servers, however I always stick to the safest and most reliable which usually happens to be the most popular option out there: httpd. Here is an optimized httpd.conf targeting a dedicated server box for running only httpd (no mysql) and Magento:

A couple of assumptions first:

  • You are running CentOS (I did not test any other distros)
  • You have httpd installed
  • You have lots of RAM (this server has 16GB of RAM)
  • This is a dedicated box
  • You know what you are doing…

Download the file, copy over your httpd.conf, restart httpd:

wget http://www.activoinc.com/downloads/httpd.conf-magento
cp /etc/httpd/conf/httpd.conf ./httpd.conf.old
cat httpd.conf-magento > /etc/httpd/conf/httpd.conf

Once done, you will also need to update the vhosts area of the file near the bottom of it. In case you wanted, here is a direct link to the file: http://www.activoinc.com/downloads/httpd.conf-magento

Web Application Hosting ,

Required PHP Extensions for Magento eCommerce

August 27th, 2009
Comments Off

When I setup a new linux server I make sure to set it up lean since performance is always on my mind. I usually install packages manually, so here is the list of PHP extensions that are needed for the latest stable version of Magento:

  • PHP
  • php-mysql
  • php-mcrypt
  • php-hash
  • php-xml (will be required for Magento 1.4.xx)
  • php-gd
  • php-pdo
  • php-mhash
  • php-soap

Here is the command line that will do the job:

yum install php php-mysql php-mcrypt php-xml php-gd php-pdo php-mhash php-soap

Tip: you should use PECL extension APC. Also known as Alternative PHP Caching. This little extension can speed things up significantly. Assuming you have setup Atomic as a repository in yum, you can run this command:

yum install php-pecl-apc

Web Development

A Quick Way to Download, Install, and Setup pdnsd (local DNS Caching)

August 26th, 2009
Comments Off

Back in December of last year I posted an article about setting up pdnsd. This is a revised version of the same howto, but with a shorter approach for fast setups. I also changed the IP addresses to point to the OpenDNS servers which has proven to be both effective and reliable.

wget http://www.phys.uu.nl/~rombouts/pdnsd/releases/pdnsd-1.2.7-par_sl5.x86_64.rpm
rpm -i pdnsd-1.2.7-par_sl5.x86_64.rpm
echo "server {" > /etc/pdnsd.conf
echo "label=\"opendns\";" >> /etc/pdnsd.conf
echo "ip = 208.67.222.222,208.67.220.220;" >> /etc/pdnsd.conf
echo "}" >> /etc/pdnsd.conf
service pdnsd start

echo START_DAEMON=yes > /etc/default/pdnsd

Edit your resolve.conf file and make sure that the first row has nameserver=127.0.0.1, the rest should remain the same. Here is how:

vi /etc/resolv.conf
nameserver 127.0.0.1

Restart your network service:

service network restart

Last: make sure that pdnsd auto starts on boot, I use ntsysv for this.

Web Development

Setting noatime and nodiratime for improved disk performance

August 25th, 2009

This is great if you have a server that performs plenty of disk access operations and you are interested in speeding things up a bit. Here is how to do this in three steps. First some assumptions:

  • You have root access
  • You are the only one that will ever need to mount or unmount this file system
  • Running CentOS (this may work, but was not tested on any other linux server)

This is what we are doing:

  1. Editing /etc/fstab to set the noatime and nodiratime flags for the file system
  2. Remounting the drives/file systems (without rebooting)
  3. Checking our work

Step 1: Edit the /etc/fstab, type:

vi /etc/fstab

Add the noatime, nodiratime flags right after the defaults flag in the “/” root mount partition. Repeat to all the partitions that you wish to speed up. Here is how my fstab file looks like after the change (click to enlarge):

etc-fstab-in-vi_with_noatime_nodiratimeStep 2: Remount the file system:

mount -oremount /

Step 3: Check our work:

cat /proc/mounts

Tip: the following command will perform a tiny benchmark test, in case you want to see how fast your hard drives perform. If you want to compare, you should perform a before and after tests.

hdparm -tT /dev/sda

Web Application Hosting, Web Development ,

Howto setup iptables for www and db(mysql) on linux

August 24th, 2009
Comments Off

Since I provide fully managed hosting services to my selective clients, I’ve been setting up iptables more than I can count. After a while I have the tendency to automate common tasks. In the case of setting iptables it can easily be done by downloading and running a shell script. So I created these two shell scripts each one targets a specific server usage: one for WWW servers and the other is for dedicated DB servers.

Here is a couple of assumptions:

  • Using a linux server
  • Has iptables installed and running
  • You have root access to the server (or enough privileges to run these commands)
  • Not a must, but I only tested these scripts on a CentOS server (and many of them…)
  • You know what you are doing!

The scripts simply clears the current settings of your iptables, adds open ports as necessary and restarts the iptables service. In the case of WWW it opens the ports 22, 80, and 443 (SSL). In the case of MYSQL it opens the ports 22, and 3306 and if you wish you can uncomment a line in there to restrict access only to a specific set of IPs. In the case of a dedicated server you may only want the corresponding WWW server to be able to access the MYSQL server. Here are the steps, I will divide them by the type of the server:

Setting iptables on a WWW server:

wget http://www.activoinc.com/downloads/iptables-www
chmod +x iptables-www
./iptables-www

Setting iptables on a MySQL server:

wget http://www.activoinc.com/downloads/iptables-mysql
chmod +x iptables-mysql
./iptables-mysql

Note: there is absolutely no warranty that this will work, it is provided with the sole hope that it may save you a few minutes or a couple of hours.

Web Application Hosting ,

CentOS 5.3 Install Essentials

August 23rd, 2009
Comments Off

When I setup a new server, I typically install it with nothing checked in the packages list of the installation process. I like using yum update first and then running yum install on the packages that I absolutely need. Clean and mean is my favorite way to run a Linux server. Two main reason are behind this: one is performance, this is a bit obvious: the less you got on the HD and processes running in the background the faster the server. Two is security: the less software you have installed your vulnerability “surface area” is smaller.

While installing it clean is great, I do have a minimum set of tools that I usually need in any server. Most of these tools are small and do not require background services so I install them almost by default. You should check if they suite your needs and use it at your discretion. Hint: the goal here is to copy and paste once a new server is installed.

(correction) Before I can use the next command I need to install wget:

yum install wget

Add the Atomic repository (newer versions of LAMP, some security packages):

wget -q -O - http://www.atomicorp.com/installers/atomic.sh | sh

Install basic packages:

yum install unzip sendmail ntsysv fail2ban logrotate pdns

LAMP: Linux Apache MySQL PHP, Web Application Hosting ,

Howto change the Base URL in Magento via command line

August 7th, 2009

A quick howto that complements chandansweb’s article on how to change the Base URL using phpMyAdmin. On production servers I do not install phpMyAdmin and therefore I need to use command line. Here is how in 3 steps:

Step 1: Login to mysql (command line) and update two records:
Note that the values in square brackets need to be filled with your specific values.

update [prefix]core_config_data set value="http://[domain]/[rootfolder/]" where path='web/unsecure/base_url';
update [prefix]core_config_data set value="http://[domain]/[rootfolder/]" where path='web/secure/base_url';

Step 2: Clear cache by removing entries in the /var/cache/* folder:
Careful with this one, make sure you are pointing to the right folder.

rm -rf /var/www/[path to html folder]/var/cache/*

Step 3: Force Magento to clear its database cache by running this php file:
Note that I recommend that you copy and paste this code in a file at your Magento root folder. Then point your browser to this file, it will trigger the clearing of various cache objects in Magento.

<?php
ob_implicit_flush(true); //Saves having to flush manually
set_time_limit(0); //Set time limit to unlimited, though we shouldnt need to
ignore_user_abort();

require_once("app/Mage.php");
umask(0);

// clean overall cache
Mage::app()->cleanCache();

// clear 'refresh_catalog_rewrites':
Mage::getSingleton('catalog/url')->refreshRewrites();
echo 'Catalog Rewrites was refreshed succesfuly<br>';

//  clear 'clear_images_cache':
Mage::getModel('catalog/product_image')->clearCache();
echo 'Image cache was cleared succesfuly<br>';

//  clear 'refresh_layered_navigation':
Mage::getSingleton('catalogindex/indexer')->plainReindex();
echo 'Layered Navigation Indices was refreshed succesfuly<br>';
?>

That should do it, you should now be able to point to your production store and it will pull the right URLs. It worked for me after various attempts of other howto’s. The above steps are the ones that worked for me.

Magento , ,

Is Magento for eCommerce Scalable?

August 4th, 2009
Comments Off

My recent experience with Magento brings a doubt: is Magento Scalable? I am finalizing a plugin that will allow catalog synchronization with a POS system. Some stores have over 10,000 products, skus to be accurate. I have extended the core classes to accommodate for the product import with certain attributes and was shocked to discover that for each new product saved, or in code terms, each $product->save() it was hammering the DB with approximately 1,000 queries. That also meant that importing 1,000 products took approximately 3-5 minutes on a powerful server which included 15K SCSI hard drives.

will_magento_scale

Writing directly to the database is not an option because it will potentially introduce a problem with any upgrade efforts and in general is not recommended. There is a limit to how much hardware is being thrown at Magento in order to cover for any performance imperfections. Additionally, almost any interaction with the Magento support team results in lists of mysql configuration best practices which are already in place. Magento: you can run but you cannot hide! Perhaps you cannot run either.

The bottom line is that I am positive that there is a solution, and I am 100% convinced that the Magento developers are working on a solution as I write this blog. It is only in the meantime that it is painful to see all these great sites serving ‘gummy’ pages and developers asking how to add the ‘Please wait while loading’ AJAX loader to the front end of the store…

Magento

Magento: Hard but not Impossible

August 4th, 2009
Comments Off

magento_hard_but_not_impossibleIn the last few weeks I have been hard at work on a Magento plugin that extends the eCommerce app to become a front end of a POS (point of sale) system. With over 6,000 files Magento can certainly be intimidating at first, but as you get familiar with its architecture you find it less and less intimidating. One main design feature makes it easy to find a class by its class names.

For example, the class Mage_Catalog_Model_Product will be located in the /app/code/core/Mage/Catalog/Model folder and the file name is Product.php. If you ever need to instantiate the class, use Mage::getModel(‘catalog/product’).

Another useful tip about the architecture is that most everything is handled in the various session objects and every operation is basically an event that occurs which then calls a certain method within a particular class. Some of these events are dispatched and can be used by Magento’s event/observer architecture but most are not and therefore one needs to override the class instead.

I know this is very vague but will make sense if you are knee deep inside the magento code.

eCommerce, Magento, Web Development

Block access to your dedicated server automatically if more than 3 failed logins

August 3rd, 2009
Comments Off

Lately I have been noticing high activity of cyberattacks. In fact, a few of our servers got hit and had to be rebuilt. Of course some of these servers were never built with security in mind. We did manage to save all the data and the redo took less than one day total so the end result is great overall with fully patched servers, firewall, email alerts in place, and finally a way to automatically block failed logins. I’ll show you how to setup the last part, it is actually quite easy.

So, I assume you have a CentOS server. I am using CentOS 5.3 fully updated (yum update) and i have the atomicorp repository setup. The atomicorp is not necessary (it seems) but will allow you to use newer versions of PHP and other packages. The package that will do the work is called fail2ban. A simple install seems to take care of it for us:

yum install fail2ban

Lets install the service, in case that we restart the server – the service will automatically start running, I use ntsysv for this:

ntsysv (hit enter, select the service, make sure it has a start inside the brackets, and click 'ok')

Lets start the service:

service fail2ban start

Here is recommend to setup a auto forward on the root emails to you:

echo 'youremail@yourdomain.com' > ~/.forward

You should be all set. Try to check the log files once a week after the install, see if the service actually blocked potential hackers. Let me know if this worked for you or if you are using a better package?

Web Application Hosting ,