Skip to content

The Magento REST API: A Better Way to Integrate Business Applications

Merchants have been asking for a fast and secure way to integrate more business applications within Magento. We’ve met this request by introducing the Magento REST API as part of the Magento Enterprise 1.12 and Community 1.7 releases.

Noteworthy benefits of the REST API include simplicity, ease of testing and troubleshooting, and better performance. It allows you to manage customers, customer addresses, sales orders, inventories and products using HTTP verbs such as GET, POST, PUT and DELETE. Data requests and responses can be in XML or JSON format.

REST Resources

REST resources are simply the entities or identities that are exposed to the developer. REST defines the identity of the resource via the URI (uniform resource identifier). Each resource has a unique URL address and any interaction with a resource takes place at its URI. The following resources are supported in CE 1.7.0.0.

  • Products: Allows you to retrieve the list of products, create a simple product, and update or delete a product.
  • Product Categories: Allows you to retrieve the list of categories assigned to a product and assign or unassign a category to a product.
  • Product Websites: Allows you to retrieve the list of websites assigned to a product and assign or unassign a website to a product
  • Customers: Allows you to retrieve the list of customers and create, update, or delete a customer.
  • Customer Addresses: Allows you to retrieve the list of customer addresses, and create, update, or delete an address.
  • Inventory: Allows you to retrieve the list of stock items and update a stock item.
  • Sales Orders: Allows you to retrieve the list of sales orders and specific order information.
  • Sales Order Items: Allows you to retrieve the items for a specific order.
  • Sales Order Addresses: Allows you to retrieve billing and shipping addresses for an order.
  • Sales Order Comments: Allows you to retrieve comments for a specific order.

Preparing to Use REST API with Magento

From the Magento store admin panel:

  • Set up permissions to operate with resources for the three different user types: admin, customer, and guest. The admin is the backend logged-in user, the customer is the frontend logged-in user, and the guest is a non-logged-in frontend user.
  • Configure which attributes will be allowed to retrieve or update for the different user types
  • Register the third-party application (setting up consumer) and provide the information to the third-party application.

For a more detailed explanation with sample data, check out magento's wiki page. As always, we welcome your feedback and are eager to help with any issues you may encounter. Please use magento's bug tracker and choose the Webservices API from the Category selection.

Magento injection vulnerability

Our staff receive an information from one of our customers about their site is being redirect to other unknown website when access any link from google search, after check their site, we found out that site is injected with some harmful php code.
Firstly, please check your index.php at magento root and find for this:

eval(base64_decode('dVFda8IwFH0X/A+XEpaGldj6AXNSpg9FXybD1b1MKV2b2GBNStpOxth/X+I+3MOEhOTec8/NuSdoXTM92zHZQAh1oxtVqiPTLkoeo9VTtHrGizh+SNYmSmbzaBnjLZl0O4K7prhStYt+G3iAd0rtSoYJKA3f+F/4LS2UuogeankRK8WraUu6nfduBwBpIxbjib0bJYiHU16rbK8qJl0c....

Or may be difference. All you have to do is remove it, usually start from line 2, remove that long line.
To be sure, please set permission for index.php to 644, then check your .htaccess for any strange line, delete all if it present in your .htaccess

Hope this help for anyone who meet this situation.

Magento speed up – Howto?

As you probably know by now, Google is Using site speed in web search ranking. And I couldn’t agree more: speed is important for your site, and not only for Google but mainly for the User Experience (UX) on your site. Research from Google and Microsoft shows that slower websites convert less and that the effect of a slow website is lasting.

Sidenote: As a psychologist this might be a form of Déformation professionnelle, but I kinda hate it when people always talk about optimizing for search engines/ Google. Don’t optimize for Google, keep in mind that you optimize for your users!

Ok, so nothing new so far (I hope), but what about the speed of your Magento platform? If you’re serious about e-commerce, 9 out of 10 times Magento will be your best choice overall when looking at features, flexibility and TOC. But there are some complaints about Magento being a very slow system and as I just explained: that isn’t good for your UX.

And although most of these complaints aren’t always fair (off course you’re Magento site is slow when you put it on a cheap shared hosting with 200 other websites!), we all know that of the box Magento isn’t the fastest boy in the class. So that’s where this list comes in…

Below is the list with 101 ways to speed up your Magento site… Well ok, at the moment I have only 30 45 54 ways, but if you help me out we can make it to 101! When new suggestions come in, I’ll update this post with additional ways to speed up your website. When more tips come in I’ll probably start categorizing them to maintain an overview.

 

The tips:

A) Hosting environment/ General tips

  1. Get a dedicated server.
  2. Host your site in the country where your customers are.
  3. Don’t host files on your web server that you do not use, large or small.
  4. Goto MySQL Admin and select all the tables and repair and then optimize them.
  5. Use a PHP accelerator like APC, ZendOptimizer+ or Xcache.
  6. Only install necessary Apache modules.
  7. Use Apache mod_expires and be sure to set how long files should be cached. You could use the example below for your Apache virtualhost config:
    # Turn on Expires and set default to 0
                    ExpiresActive On
                    ExpiresDefault A0
    
                    # Set up caching on media files for 1 year (forever?)
    
                            ExpiresDefault A29030400
                            Header append Cache-Control "public"
    
                    # Set up caching on media files for 2 weeks
    
                            ExpiresDefault A1209600
                            Header append Cache-Control "public"
    
                    # Set up 1 week caching on commonly updated files
    
                            ExpiresDefault A604800
                            Header append Cache-Control "proxy-revalidate"
    
                    # Force no caching for dynamic files
    
                            ExpiresActive Off
                            Header set Cache-Control "private, no-cache, no-store, proxy-revalidate, no-transform"
                            Header set Pragma "no-cache"
  8. Enable Gzip Compression in htaccess.
  9. Compress output, use zlib.output_compression or mod_deflate.
  10. Use a Content Delivery Network (CDN) for parallel transfer of static content. There is a Magento extension that can help you do this with category and product images: the One Pica Image CDN. But… (see next tip).
  11. Don’t use too many different external sources (for images, iframes, (twitter)feeds etc.) because every DNS lookup takes extra time.
  12. Enable Apache KeepAlives: Make sure your Apache configuration has KeepAlives enabled. KeepAlives are a trick where multiple HTTP requests can be funneled through a single TCP connection. The setup of each TCP connection incurs additional time, this can significantly reduce the time it takes to download all the files (HTML, JavaScript, images) for a website. More info  at Apache.org.
  13. Minimize redirects.
  14. Make your output W3C compliant. Errors slow down the browser.
  15. Swap Apache for NginX or Litespeed. If this isn’t an option, then go for Zend Server (ideally the non-CE version so you can use full page caching).
  16. Turn off or at least reduce web server logging (reduces disk writes).
    • Disable Access Time Logging. Magento Wiki: For Linux servers, if you have access-time logging enabled on any of your mysql, web server or cache partitions, try turning it off for a performance boost. If you’re using ext3 or reiserfs there may be faster journal write methods you can use. For more info see Linux.com.
  17. Compile MySQL from source instead of your OS’s package manager.
  18. Always upgrade to the latest Magento version. Not only will you get more features and bug- and security fixes, but with every update Magento performs better.
  19. Query Cach size: Magento Blog: Modify the configuration for your MySQL server to take better advantage of your server’s RAM. Most Linux distributions provide a conservative MySQL package out of the box to ensure it will run on a wide array of hardware configurations. If you have ample RAM (eg, 1gb or more), then you may want to try tweaking the configuration. An example my.cnf is below, though you will want to consult the MySQL documentation for a complete list of configuration directives and recommended settings.
  20. set ‘php_value memory_limit 128M’ in your php configuration or add it to your .htaccess file to ensure you don’t run out of memory.
  21. Use a memory-based filesystem for dynamic data. If you store dynamic data (var/cache, var/session) on RAMdisk or tmpfs, the disk I/O is decreased.
  22. Change realpath_cache_size in php.ini.
    realpath_cache_size=1M (careful, this is per apache process) realpath_cache_ttl=86400 (ok for production site) 
  23. Memcache (for the hardcore) is documented in http://www.magentocommerce.com/boards/viewthread/9037/ and more tips from http://alexle.net/archives/275 to get you up and running.
  24. Disable the PHP open_basedir directive. Read this.
  25. Eliminate directory structure scans for .htaccess files.
  26. Recommended innodb_buffer_pool_size.
    • Combined web and db server, 6 GB RAM:  2-3 GB
    • Dedicated database server, 6GB RAM: 5 GB
    • Dedicated database server, 12 GB RAM: 10 GB
  27. innodb_thread_concurrency.
    • 2 * [numberofCPUs] + 2
  28. Query Cach: query_cache_size: 64MB, query_cache_limit: 2MB
  29. Use sa seperate backend server to handle admin users, process backend activity (cron), pre generate full page caching and to handle media queries.
  30. Use multiple web nodes (frontend servers) to handle browsing and checkout.
  31. Use Varnish reverse proxy caching, Magento explanation by @alistairsteadVarnish your Magento store, make it fly!.

B) Template

  1. Optimize all your (template) images– Most if not all should be at least below 10kb.
    • Crop the white space using your image editor.
    • Use PNG8 Files or GIF files rather than Jpegs and don’t use transparency (depending on how many colors you use and how large the image is, but try for yourself).
    • Scale images: make images in the dimensions you need and not resizing them in the editor.
    • Use image compression (you can use smush.it to do it for you).
    • Use CSS Sprites, there even are CSS Sprite Generators.
  2. Minify your Css, remove unused code.
  3. Minimize Javascript use.
  4. Use a lightweight template as a basis for your template. For example: the Yoast Blank SEO theme.
  5. Specify Image dimensions.
  6. Use Block cache and HTML output in your extensions.
  7. Apply Javascript Lazy Loader for prototype.

C) Magento configuration

  1. Uninstall any extensions that you don’t actually use.
  2. Disable modules that you don’t use: System -> Configuration -> Advanced -> Advanced.
    modules disable 101 ways to speed up your Magento e commerce websiteThis is an example setting
  3. Enable all Magento Caches: System -> Cache Management.
    Magento Caching 101 ways to speed up your Magento e commerce website 
  4. Use an offsite Stats Tracker like Google Analytics and not an onsite one. Most of this will use Javascript, host the Javascript yourself.
  5. Combine Javascript and Combine CSS files: System ->Configuration ->Advanced ->Developer -> ‘Javascript settings’ and ‘CSS Settings’. You can also consider using an extensions to do this like the Fooman Speedster extension, whichever works best for you.
    javascript css 101 ways to speed up your Magento e commerce website
  6. Try some of the Magento performance extensions.
  7. Enable the Magento Flat Catalog, only helpful for very large catalogs to feel the improvements.
  8. Don’t use layered navigation if you don’t really need it, it’s resource intensive.
  9. Use Magento’s Compilation feature. It’s reported to give you a 25%-50% performance boost: System > Tools > Compilation.
    magento compilation 101 ways to speed up your Magento e commerce website

    Edit: Yoav Kutner (Magento CTO) let me know that “in later versions since we optimized the catalog EAV, Magento Compilation is not really needed if you have byte code caching and if it is configured correctly“.
  10. Use the correct session storage, choose file system or database (during setup). Most installations should use “file system” because it’s faster and doesn’t cause the database to grow. But if your site will run on multiple servers, you should select “database” so that a user’s session data is available regardless of which server his/her request is served from. More info about this from Ashley Schroder at Magebase.com.
  11. Limit the number of products on a product overview page.
  12. Set only those attribute frontend properties to ‘Yes’ that you’re actually going to use. Set all other to ‘No’. Don’t use in quick search, advanced search compare, etc etc.: Catalog -> Attributes -> Manage Atributes -> Frontend Properties.
    attributes frontend properties 101 ways to speed up your Magento e commerce website 
  13. Disable the Magento log: System -> Configuration -> Advanced -> Developer -> Log Settings (default is disabled).
    logging 101 ways to speed up your Magento e commerce website 

Enterprise only tip:

  • Disable Enterprise_CatalogEvent. Go to Admin -> System -> Configuration -> Catalog -> Catalog Events.
    Then you want to turn off the settings for “Enable Catalog Events Functionality” and “Enable Catalog Event Widget”.
  • Enable Solr search, it’s quicker compared to the default setup, especially when you have lots of products (>10k).
  • Enable Full Page Caching.

D) Speed testing, analysing, monitoring

  1. Test your Magento site with Magento Speed Test (by Ashley Schroder)
  2. Run your site through websiteoptimization.com.
  3. Use Google Page Speed Firefox extension or Yahoo Yslow for some tips from Google and Yahoo.
  4. Implement Google Speed measurements in Analytics: Measure Page Load Time with Site Speed Analytics Report
  5. Speed monitoring and downtime alerts.

Bonus Tips

(because it doesn’t actually speed up the frontend but only the backend):

  • Use K-Meleon if you are on Windows for your general Admin work. It renders Magento’s heavy JS back-end significantly faster than any other browser.
  • Use the GoogleGears extension from Yireo.com to cache static files locally.
  • Use a local pc/mac application to manage Magento (like mag-manager.com).

I do realize that you probably can’t use all of the above tips but it’s not about using them all, and sometimes you just make the choice to give a functionality a priority over performance and that’s OK. If you can only apply some of them you will still increase speed and gain in user experience. And remember: because speed depends on many variables, some of the tips might not have an impact on your website at all while others can have a huge impact. So always test the impact of every step you take and take actions accordingly.

Do you have any tips? Post them in the comments and I’ll add them to the list!

Sources:

Identifying Magento Performance Problems with the Magento Profiler

The Magento Profiler is used to identify performance problems on the server side.  The Profiler can help you find PHP functions which use up too much CPU or functions with slow database queries.

These problems will first be noticed if you have high load on your server. Apache processes can be seen using “top”, where you will see apache or httpd processes jumping to the "top" using a large percentage of CPU.

Using the Profiler requires a fairly deep (e.g. time consuming) analysis, so make sure you are barking up the right tree before proceeding with this. You’ll want to eliminate any front-end issues (such as loading large png files, too many css or javascript files, content compression, unnecessary Javascirpt, etc.) to be sure your problem is really server side (The “YSlow” firebug plugin is a good resource for client side problems).

Magento is very resource intensive, and many shared hosts will not be able to run it with decent performance.

Make sure your problems are not related to your database. Login to MysQL. Run show “process list”, as you go browse through the slow areas of the site. If any queries stay on the screen as you watch, you probably have a database performance problem.

Finally, make sure your problem is not a networking relating issue, such as a slow or faulty internet connection, or firewall.

Generally, look for the first page hit using Firebug Net view to see the total server side time required to generate the page. The Magento profiler is limited to this first page hit – so make sure you know how much performance you can actually gain. Focus on the greatest performance as a percentage of the overall time to view the page, to be sure you are getting low-hanging fruit first.

Be sure you know what's going on with your cache. If you are using caching, the difference between the first and subsequent hits can be huge, and will throw confusion into the mix, giving you meaningless results. I recommend adding some comments using PHP error_log() function, (tail -f the web error log), so you know when the Full Page Cache is used. See my previous blog post on the Full Page Cache.

As with many Magento problems, I’ve given up trying to find documentation or explanations online.  Although the architecture is technically beautiful and the code very well written, documentation can be very spotty. You can occasionally strike gold on the community site, but I’ve found the most direct way to approach many Magento problems is to read the source. Once you go through the code, you find out that the feature wasn’t as complicated as you thought it was. It’s this way with the Magento Profiler.

However, in this post, I’ll try to save you some pain reading the source, by sharing some experiences on how I’ve used the Profiler.

I’m assuming your running Magento Enterprise 1.8.

First, enable the profiler via System -> Configuration -> Developer -> Profile (yes). This enables the profiler, but does not fill in any of the benchmark times.

Comment out the following line in /index.php

# toggle this to enable profiler.
Varien_Profiler::enable();

Next, refresh the page you are optimizing. At the bottom of the page, you will see the performance table.

Magento Profiler at Bottom of Page

Fig 1: Magento profile data at bottom of page.

This table is impossible to read directly inline, since the HTML is placed outside of any HTML or body tags. Go into the source and copy the entire page contents. Paste in Notepad or any other text editor. Eliminate the regular page HTML , leaving only the HTML which builds up the performance data table. Save to a temporary HTML file and then view via Internet Explorer to view the static page. (IE allows you to copy the table from HTML to Excel)

Fig. 2: Magento stats loaded into Excel. (Click to enlarge.)

The call to Mage::app should be at the top of the list. This is the full time of your request is taking to run on the server (minus process startup). It’s what you want to reduce as much as possible. The code found in app/Mage.php is what "marks" the start and end points to profile.

Varien_Profiler::start('mage');
... core magento code ...
Varien_Profiler::stop('mage');

I Ignore the memory usage stats. (If you’ve figured out how to make these egregiously large numbers have any meaning, leave me a comment.) Under normal conditions, Magento chews up around 50 Megs of memory per process. If you are running data loading  scripts, it can use up much more memory if there are repeated instantiations of Magento objects (users, products, etc).

The number of instantiations is very meaningful, as it will tell you if there are unnecessary objects) being made, possibly through some customizations you've made. But don’t assume the Magento code is perfect either. We found out that any more than 10 shopping cart rules will slow the performance of the cart to a crawl, due to repeated calls to EAV load table. (NOTE: EAV load calls are very expensive, performance wise. Each one costs about 1/10 of a second. I'll post another blog article on that solution if it will help someone – let me know.)

The column "Time" indicates the total time spent between the "start" and "stop" calls within "Cnt" instantiations. To resolve your performance problem, look for large numbers of instantiations, resulting in large "Time" values within this report.

Use recursive grep on the source to find out what is being measured within the profile report. E.g.

grep -r "Varien_Profiler::start('mage'" *

Also, you should be able to add your own Varien_Profiler::start() and stop calls within the code (though I haven't done that).

Good luck and I hope this article helps with troubleshooting your Magento performance problem. Leave me a comment if you have more information or need some help.

Fix 404 error when access admin panel after upgrade or fresh install

Some time, you meet this 404 error after upgrade you magento website, or even fresh installation. Your frontend running smooth but you can not access your backend. Do the following ways to bring your backend accessible. One at a time, if problem fixed, it's not necessary to do anothers.

  1. Clean your cache by manually remove var/cache dir content.
  2. Check your local.xml (app/etc/local.xml). Look for following line:
    
        
            
                
                    {{admin_frontname}}
                
            
        
    

By default frontName hold this value:

If changed, please change it back to:

Clean cache by manually remove var/cache dir content.

  1. Check your var/log/system.log for error log. If you can see following error log:
2011-10-14T18:59:31+00:00 ERR (3): Recoverable Error: Argument 1 passed to Mage_Core_Model_Store_Group::setWebsite() must be an instance of Mage_Core_Model_Website, null given, called in /home/ngoc/www/m1600/app/code/core/Mage/Core/Model/App.php on line 644 and defined  in /home/ngoc/www/m1600/app/code/core/Mage/Core/Model/Store/Group.php on line 235
2011-10-14T19:18:35+00:00 ERR (3): Recoverable Error: Argument 1 passed to Mage_Core_Model_Store::setWebsite() must be an instance of Mage_Core_Model_Website, null given, called in /home/ngoc/www/m1600/app/code/core/Mage/Core/Model/App.php on line 624 and defined  in /home/ngoc/www/m1600/app/code/core/Mage/Core/Model/Store.php on line 304

If you see this error message, please make sure that you can access your database to execute this SQL command:

SET FOREIGN_KEY_CHECKS=0;
UPDATE `core_store` SET store_id = 0 WHERE code='admin';
UPDATE `core_store_group` SET group_id = 0 WHERE name='Default';
UPDATE `core_website` SET website_id = 0 WHERE code='admin';
UPDATE `customer_group` SET customer_group_id = 0 WHERE customer_group_code='NOT LOGGED IN';
SET FOREIGN_KEY_CHECKS=1;

Result will be:

SET FOREIGN_KEY_CHECKS =0;# MySQL returned an empty result set (i.e. zero rows).
UPDATE `core_store` SET store_id =0 WHERE code = 'admin';# 1 row(s) affected.
UPDATE `core_store_group` SET group_id =0 WHERE name = 'Default';# 1 row(s) affected.
UPDATE `core_website` SET website_id =0 WHERE code = 'admin';# 1 row(s) affected.
UPDATE `customer_group` SET customer_group_id =0 WHERE customer_group_code = 'NOT LOGGED IN';# 1 row(s) affected.
SET FOREIGN_KEY_CHECKS =1;# MySQL returned an empty result set (i.e. zero rows).

Clean cache by manually remove var/cache dir content.

Last error is popular for many developer. Hope everyone can fix your error and bring back your magento backend again.



					

Magento – WordPress Integration. The easiest way

Still searching a best way to intergrate your magento with wordpress, you can have it done in a few small steps, no more headache. Try Fishpig’s Magento/WordPresss Integration Extension, and you can make your blog run with magento.
In this tutorial, we have:
– Magento installed in root folder (e.g., /public_html/)
– WordPress installed in a subfolder (e.g., /public_html/wordpress/)
– Two software installed on separate database
First, you need to install Fishpig extension. Grab it by click here. After successful install, you can see a new menu in your admin dashboad. Follow menu:
 

WordPress>Settings>Database / Integration

Config your database settings:

Database Location: Magento/WordPress share a database
Host: localhost
Username: your_wordpress_mysql_username
Password: your_wordpress_mysql_password
Database Name: your_wordpress_mysql_database
Database Charset: utf8 (leave default)
Table Prefix: wp_ (leave default if you did not change table prefix when install your wordpress)

Config your integration settings:

Integration Level: Fully integrated
Blog Route: blog (you can use another name. i.e. news)

Remember that your Blog Route can not be same as your wordpress directory

Save all change,  And now you can access your blog under magento. Note that you need to re-style your blog cos integration use it own template.
 

Part 8 – Varien Data Collections

Originally, as a PHP programmer, if you wanted to collect together a group of related variables you had one choice, the venerable Array. While it shares a name with C's array of memory addresses, a PHP array is a general purpose dictionary like object combined with the behaviors of a numerically indexed mutable array.

In other languages the choice isn't so simple. You have multiple data structures to chose from, each offering particular advantages in storage, speed and semantics. The PHP philosophy was to remove this choice from the client programmer and give them one useful data structure that was "good enough".

All of this is galling to a certain type of software developer, and PHP 5 set out to change the status quo by offering built-in classes and interfaces that allow you to create your own data structures.

Part 7 – Advanced ORM – Entity Attribute Value

In the first ORM article we told you there were two kinds of Models in Magento. Regular, or "simple" Models, and Entity Attribute Value (or EAV) Models. We also told you this was a bit of a fib. Here's where we come clean.

ALL Magento Models inherit from the Mage_Core_Model_Abstract / Varien_Object chain. What makes something either a simple Model or an EAV Model is its Model Resource. While all resources extend the base Mage_Core_Model_Resource_Abstract class, simple Models have a resource that inherits from Mage_Core_Model_Mysql4_Abstract, and EAV Models have a resource that inherits from Mage_Eav_Model_Entity_Abstract

If you think about it, this makes sense. As the end-programmer-user of the system you want a set of methods you can use to talk to and manipulate your Models. You don't care what the back-end storage looks like, you just want to get properties and invoke methods that trigger business rules.

Part 6 – Setup Resources

On any fast paced software development project, the task of keeping the development and production databases in sync become a sticky wicket. Magento offers a system to create versioned resource migration scripts that can help your team deal with this often contentious part of the development process.

In the ORM article we created a model for a weblog post. At the time, we ran our CREATE TABLE statements directly against the database. This time, we'll create a Setup Resource for our module that will create the table for us. We'll also create an upgrade script for our module that will update an already installed module. The steps we'll need to take are