Moving data around can be a lot of work. Especially when you have to do it manually. Let’s say you have a webshop with a lot of products, modifying all products by hand can take days. For a client of us we developed a fully automated product import system with Laravel. We got asked to automate the entire process from reading different product feeds to pushing these products to the webshop.
The first step was to analyse all the different product feeds. All these feeds have different structures. They also contain different types of products. On top of that, we needed to be able to read multiple file types, including XML and JSON.
Our first challenge was to be able to read all the different feeds, so we could combine them and push them to the webshop.
Once we were able to read these feeds, the next step was to store the data in our local database. Feeds contain multiple products and every product needs to run separately. Why? Well, when one product fails because of an error, you don’t want the entire feed to fail. For this we make use of Laravel Queues. Thanks to Laravel Queues we are able to run every product individually.
After all the products are imported to the local database, the products can be exported to the Woocommerce webshop. Before that, we need to decide which products we want to export. This is where product funnels come in. These product funnels allow us to filter our products by title, category, or every other meta field. This is extremely helpful when we want to update some products more often than others. Of course, product funnels are optional.
Thanks to Laravel Horizon, scheduling tasks is easy to do. You simply set up a queue. Queues make it possible to chunk up big tasks in multiple small tasks and schedule them after each other. Within Horizon you can easily see the status of all the jobs that have been deployed.
In our automated import system, every product is deployed as a single job. This is the safest way to export products because when one product fails to export, all the other products will continue.
All the products that have been selected for the export will be pushed to Woocommerce. To do this we use the Woocommerce Rest Api. This api allows us to create and update products inside an existing Woocommerce Webshop. We can also create categories, product attributes and product variations.
All the data for the export is stored in our local database. The data is formatted in the same structure as the Woocommerce Rest Api, this way Woocommerce knows all the fields. This data is stored in our local database so we can split up all the tasks in multiple queues. First we check if all the product attributes exist. If they don’t exist, we create them. Next we check if the categories exist, so we can link all the products to the correct category. We also check if a products exist. If a product exists, it will be updated.
Currently I am working on some other Laravel projects. If you want to know more about this, feel free to contact me on Twitter