Guides

Catalog V2 vs V1

Suggest Edits

More and more new customer are trust on Instaleap e-commerce solution, in addition to the constantly growth of our old customer, more information is being processed in our platform. As a result, we saw necessary have a new implementation with more resources, optimizations and stand alone machines per customer to keep growing with our customers and remain be that trustworthy ally.

Improvements

  • Latency, throughput and Performance: The current version of catalog does not allow optimization to improve the general performance of API services. This means that customers in API-V1 will not be able to update the catalog with the speed and loads expected for the business challenges for 2022. Catalog V2 will be a Reactive system: resilient , fault tolerant and with the elasticity to process extremely high loads. Notice that the first iteration Catalog V2 has 10X more power than Catalog V1 to process data.

  • API implementation: The API-V1 has several unfavorable characteristics from both technically and business point of view:

    • Async response: The Bulk API in Catalog-V1 does not guarantee the creation or update of products during the requests processing. As a consequence, once a Bulk request is received, the delivered response does not contain the result of the operation in DB. This means that the "successful" response in the Bulk service doesn't means that the process was successful, on the contrary, may fail in the writing process. To avoid this lack of reliability and always assure quality, Catalog V2 only have Single API's with highly resilient and power to support the volume of data of a Bulk API with better performance according the load expected of our clients in coming years. This means that customers can sent a high volume of data through Single API's (multiple requests) and this will be fast enough to replace the Bulk Api with the improvement that the Single API responds if the process was successful or not.

    • Duplicate records: During the processing of Bulk request, it's possible that duplicates may be created in particular situations. This problem gone in Catalog V2.

  • Loads distribution: with the constantly increasing of data processing in our systems, a customer high load can generate a service degradation in other customers. Catalog V2 implemented a system with decentralized DB per customers, so that the consumption of one customer does not affect the others.

  • Data structuration: The current data structure does not handle the independency of general attributes (doesn't change among stores) of products like sku, description ,brand, etc and the catalog particular attributes (change among stores) like, price, stock, maxQty, etc. As a consequence, the customer is required to send the same information every-time a create new catalog for a store is needed . Catalog V2 consider an efficient data structure for both us and the customer.

  • New Features: Catalog V2 is thinking to be clean and scalable, so it allow us to create new features faster. Please notice that new features will be only available Catalog V2. This applies also to small bugs the current system has.

For more information about the technology used in Catalog V2, consult The Reactive Manifiesto

Differences between API-V2 and API-V1

Create products per store in V1 VS Create products and Create Catalog per Store in V2

In Single API-V1, the customer needs to create each product multiples times if he wants to have it in multiples stores. This means that the attributes like name, descriptions or photos must repeated by product per store, despite it will always be the same information. In API-V2, the attributes that don't change per store were separated in one different collections from those that do.

Differences:

  • In Catalog V2, the endpoint to create a product is different from the endpoint to update a product. In Catalog V1, it's the same endpoint to create and update a product (an attribute calling "upsert" was used to differentiate when a product was updated or created).

  • In Catalog V2 exists two collections, one for products that contain the general information of products and other calling "Catalogs" that associate those products with the stores and contain the information that change among stores like stock and price. The advantage of having two collections is don't having to create the same product several times to have it in each store. In Catalog V1 only exist the collection of products and the customer must to create the products several times (one per store).

  • To create one product in Catalog V1, the customers needed to sent the mandatory attributes in the products attribute: name, SKU, photosUrls, unit, price, stock, ean and categoryReference (the others attributes aren't mandatory to create a product). To create one product in Catalog V1, the customer only need to sent the following attributes: name, photosUrl and SKU (the others attributes aren't mandatory to create a product and will have default values).

  • In Catalog V2 to create a product in one store, the customer needs to send the mandatory attributes: productId (created at the moment to create a product and is the key to associate one product), storeId (created at the moment to to create a store), price, stock, categories (id of categories, can be one o more), maxQty, minQty, isActive and location. In Catalog V1 the customer needs to create a complete product.

Payloads:

  • Payload to create a product in V1 ("upsert" is set in false to update a product)
{
"storeReference": "string",
"upsert": true,
"product": {
"name": "Chocolate chips",
"description": "Still using disposables? It's time to switch to a real razor.",
"sku": "012345",
"photosUrls": "photoUrl1, photoUrl2",
"unit": "string",
"showSubUnit": true,
"price": 0,
"stock": 0,
"ean": "ean1, ean2",
"categoryReference": "categoryReference1, categoryReference2",
"formats": []
}
}
  • Payload to create a product in V2
{
     "photosUrl": [
          "https://res.cloudinary.com/walmart-labs/image/upload/v1591796520/gr/images/product-images/img_large/00750301926874L.jpg"
     ],
     "ean": [
          75030192742
     ],
     "name": "Mezcla especial calidad selecta 125 g",
     "sku": 7503016874,
     "unit": "pz",
     "showSubUnit": false,
     "clickMultiplier": 2,
     "subUnit": "gr",
     "boost": 9999999,
     "description": "description of product",
     "brand": "brand",
     "search_keywords": "key",
     "subQty": "500",
     "nutritionalDetails": "nutritional"
}
  • Payload to create a Catalog in V2
{
     "categories": [
          "3321fe5d-d728-401b-93e9-040e81223676",
          "3321fe5d-d728-401b-93e9-040e81223575"
     ],
     "productId": "7e1f6818-507a-4956-b978-d2fa43806238",
     "storeId": "19d753e3-1235-4081-b794-59dcf971d928",
     "price": 125010,
     "stock": 100,
     "maxQty": 10,
     "minQty": 3,
     "isActive": false,
     "location": "Warehouse 1",
     "securityStock": 10
}

Create/update products by stores VS Create/update products and Create/update Catalog by Store

In Catalog V1, the customer needs to use the same endpoint to update the products (for update, "upsert" is set in false). Catalog V2 has independent endpoints to create and update products and catalogs (only the id of the product or catalog is require as query params, and it's possible modify all attributes except "productId" and "SKU" for products and "productId" and "catalogId" for catalogs).

Create or update multiple products by store

As It mentioned above the first difference is the deprecation of Bulk API that allowed you to create or update multiple products in one request. This is due the problems in performance, async response and duplicate records and was replaced by Single API with the capacity to process requests 10X faster.

how to create or update multiples items in V2:

  • To create multiples products/catalogs in API-V2, you must use:

    1. the endpoint to create products the times that you need to create all products that you want using a loop that send them one by one (apply also to create catalogs). The API-V2 is faster enough to response and receive higher amount of requests.

  • To update multiples products/catalogs in API-V2, you must use:

    1. the endpoint to get products to know the id of all or selected products.

    2. then, use the endpoint to update products the times that you need to update all products that you want using a loop that send them one by one (apply also to update catalogs). The API-V2 is faster enough to response and receive higher amount of requests.

  • To update multiples catalogs in API-V2 and ignore the catalogs that could be in DB but don't have information to update ("othersOutOfStock" function in V1), you must use:

    1. the endpoint to get catalogs to know the id of all catalogs.

    2. Then, use the endpoint to update catalogs several times, sending all the products (both those that I want to update and those that I do not want to be displayed that I previously obtained with the Get) using a loop that sends the catalogs one by one. The Catalogs with a product associated that you want to display in the ecommerce must sent to update the attribute "isActive" in true and the other catalogs must send in false for don't show it .The API-V2 is faster enough to response and receive higher amount of requests.

Updated about 2 years ago