Hot Folders in SAP Commerce(Hybris)

Hot Folder is one of the best way to import and feed data into Hybris. It serves as staging area which is continuously monitored and when files are copied or dropped into it, they are automatically processed.

It is based on Spring integration framework.

Why we use Hot folder?

• automize data import;
• fast than regular impex;
• files can be generated by other system.

With Hot folders we transform .csv files using ImpexConverters to impexes and import them using ImportService.

Spring integration periodically scans the configured input directory for new files. If new files are found, they are moved to the processing subdirectory and then sent to the Batch Import pipeline, which consists of the following tasks:

• HeaderSetupTask: Creates a new BatchHeader with some information. BatchHeader is used as a reference for file throughout the flow.
• HeaderInitTask: Initilization step which helps us to retrieve sequenceId and (optionally) a language from the file name and add them to the BatchHeader.
• ImpexTransformerTask: Creates one or many impex files from the original CSV file with the help of ImpexConverter and writes error lines to the error subdirectory.
• ImpexRunnerTask: Processes(imports) all impex files sequentially with multiple threads using ImportService.importData().
• CleanupTask: Deletes all transformed files(impex) and moves the imported file(original csv) to archive folder.
• ErrorHandler: Deletes all transformed files(impex) and moves the imported file(original csv) to error folder.

To configure hot folder we need to define a base directory where the csv files will be put, initiate our flow with the catalog and base director, and create an ImpexConverter and associated with a MappingConverter.

Now, i will show you example hot-folder-example-spring.xml file and explain what it should contain.

1. We create hot-folder-example-spring.xml in \hybris\bin\custom\example\examplecore\resources\examplecore\integration and import it in examplecore-spring.xml

in examplecore-spring.xml

in hot-folder-common-spring.xml we will define our exampleImpexProductHeader later.

2. We add Config base directory bean in hot-folder-example-spring.xml to specify where our files will be put and scanned.

3. We add inbound-channel-adapter to watch files inside the base directory with names matches the pattern specified with filename-regex.

<int:poller fixed-rate> — is used to configure periodic trigger to sniff the files.

comparator=”fileOrderComparator” — this bean specifies the order of the processing. It’s Hybris standart comparator but we can customized it extending FileOrderComparator.java.

3. Add outbound-gateway to move the file to processing

4. Add service-activator to feed the flow with relevant information.

Let’s say we want to initialize the batch header not with the sequence id and the language but with date(time) and language. How we can do that?

We can extend HeaderInitTask.java and override execute() method.

5. Add transformer converter mapping.

The batch Transformer Task collects every mapping (i.e. every bean that implements the ConverterMapping interface) and uses it for mapping the right converter. There is no need for further configuration; just define a mapping bean in the Spring context.

6. Then add specific Converter.

ImpexConverter has two properties:

• header : the header of the generated impex;

{exampleImpexProductHeader} — is defined in hot-folder-common-spring.xml.

• impexRow : the mapping between the column of the csv file and the impex file.

{+0} means that column 0 will be mapped to code[unique=true] and etc.

The ‘+’ character adds a mandatory check to this column. Any lines with missing attributes are written to an error file in the error subdirectory.

The ‘S’ -{S} can be used for writing the current sequence ID at the template position. Optionally, columns can be quoted by enclosing the column in the template with quotation marks.

• rowFilter: An optional row filter. The supplied expression must be a valid Groovy expression. The current row map consisting of column ID and value is referenced by row.
• type: An optional type that can be retrieved in the header using the header substitution $TYPE$.

Hope this article helps you to understand better Hot Folders in SAP Commerce(Hybris) :)

You can learn more: https://help.sap.com/viewer/4c33bf189ab9409e84e589295c36d96e/1811/en-US/8ad0ddef866910148b9ba21d1ffa31cf.html