Proper integration of 1C with other systems

One and a half million Russian companies use one or more 1C products for management and accounting, project and product management, enterprise management, procurement and sales.

As a rule, 1C is not the only product in IT architecture, so for most of these companies, the issue of data exchange between applications in the IT circuit is relevant. In this article, we'll show you how to properly build integrations between 1C programs and applications from other vendors in order to avoid data conflicts and loss of information.

Features of data organization in “1C”: a problem with many variables

The storage system in any of the “1C” programs is designed in the form of a multi-level complex database. It spells out the relationships and interdependence of data, their connections with objects with a high degree of abstraction.

However, in our experience, companies rarely use “1C” in the same form and in the same configuration that is provided to the user initially, “out of the box”. Each company refines systems for itself, for the specifics of its business processes, organizational structure, manufactured products, etc.

As a result, the data structure becomes even more specific and unique.

As long as all this is spinning within the same system, there are no problems. But if you need to integrate 1C with other systems, the specifics of storing information complicate this task.

There is a need to understand the following nuances.

1. Which system will the integration itself include: 1C or the second “participant in the process”? This will determine which development team will create and revise the integration and which system will be loaded.
2. In what format should data be sent and received? Different systems consume data in different formats. “1C:ERP Enterprise Management” is in EnterpriseData (based on XML), and, for example, Axapta is also in XML, but in “pure” XML. The differences seem to be minimal, but they can affect data quality. And either in the receiver system or in the data source system, you will need to configure the conversion to the desired format.
3. What data should be collected and sent? As a rule, the receiver system does not need “everything, everywhere and at once” — only individual fields or records for a certain period of time. It is necessary to determine what these fields are, separate them from the general data array and send them to the appropriate receiver slots.
4. Which entity attributes should be passed to and from 1C? And what if the final system doesn't have the right value? For example, nomenclature attributes can be stored in 1C as several directories that are updated independently.

For example, you send data to 1C:Retail on the “table lamp” nomenclature, in which the “color” directory attribute value is “red”.

If “1C:Retail” is not yet set to “red” in the color guide, it will be impossible to record such data.

All this should be taken into account already at the stage of initial integration development. But that's not all. 1C regularly releases and updates its product code. The data organization for which you wrote your integration may become irrelevant after any of the updates. The field names will change, some fields will be split into two or combined. And all integrations related to 1C will have to be redesigned in accordance with new circumstances.

There may also be changes on the side of the second system involved in data exchange, and the integration will have to be changed again.

Options for integrating 1C and other applications

By and large, there are only two options for integrating 1C with applications in the IT circuit: direct integrations and integrations via the enterprise service bus, or ESB (short for enterprise service bus). Each of these approaches has its own peculiarities.

Direct integrations

The first (and more preferable, as practice shows) option for implementing direct integrations is to work through the universal OData protocol (abbreviated from English open data protocol). 1C products, in particular 1C:Enterprise version 8.3.5 and newer, are preinstalled with the ability to work via OData, an open web protocol for requesting, adding, deleting and updating data. OData allows you to perform operations with resources using HTTP requests and receive responses in XML and JSON formats.

This is a universal protocol that is used not only by 1C, but also by other systems. Therefore, the systems in the circuit will more easily “understand” the data obtained.

To use the OData protocol, you need to enable its support in the “1C” settings. After that, the system will automatically create a REST interface for data exchange between 1C and other systems. And it will already be possible to “attach” the integration to it and configure the upload and download of data.

If the system is older than version 8.3.5 (for example, 8.2), you will not be able to use the universal OData protocol. You'll have to write an API and integration for each of the connected systems.

It would seem, what's the catch? The practice of writing an API is a standard solution for setting up “communication” between two systems.

But here we should take into account that people write the API. Moreover, people are different, with different experiences and different logics. They can even solve the same problem in different ways.

As a result, if, in addition to 1C, you have a dozen systems integrated with it and several “1C” developers, you are highly likely to own a “zoo” of ten or more APIs. Each one of them is:

• written in its own logic;
• works according to its own logic;
• It should be improved in the future... yes, again in its own logic.

After each “1C” update, your team will have to separately refine each API in accordance with the latest changes and the internal logic of the API. At best, the same specialist who developed the API will do this; it will be easier for him to recreate his way of thinking when writing code. At worst (in terms of time), he's a completely new specialist who practices different approaches to writing an API.

The zoo problem can be solved by introducing strict API writing standards (although this does not always work in practice) or by artificially reducing the number of integrations.

We saw an example of such a reduction in one of the projects. To reduce the “zoo” in the “1C” source (the system in which data on suppliers' goods were saved), the company's developers implemented chain integration.

Bitrix collected information from other systems about balances, prices, goods, etc. and transferred it to 1C, a source. The 1C source transferred all product data to the 1C system, which is responsible for offline retail. From it, some of the initial information went to 1C, which is responsible for the online showcase.

On the one hand, this reduced the number of integrations as such. On the other hand, each next link made the circuit as a whole more vulnerable and gave more room for mistakes and losses. After all, any failure in the integration or in the link system led to the information getting stuck and not reaching the next stages.

CVs

• “1C” can be directly integrated with other systems via the OData protocol or through the API.
• The OData protocol is universal and makes it easy to exchange data between 1C and other systems.
• By default, OData is only available in relatively new versions of 1C.
• Writing an API in 1C is not regulated. Each developer implements it based on their own experience, logic and approaches.
• After updates to 1C and other systems, all APIs will have to be changed. If they are written in different logics, the task becomes more complicated.
• Reducing the number of integrations by complicating the logic of system interaction actually makes the entire circuit more vulnerable to errors.

1C integration via a data bus

If the company has only two applications (for example, 1C:ERP Enterprise Management and WMS), you can choose to integrate the point-to-point format. You know exactly what data and how much data is stored in 1C, what data is stored in the inventory control system, what each system should transfer and what it should keep. Global growth is not planned in the coming years, as is the introduction of other IT systems.

But if the IT architecture includes more systems, more entities are moving between them, and growth is planned in the foreseeable future, it is better to look for fundamentally different solutions. Those that will be easy to maintain and that will not require the development of something unique every time the company adds new systems to the IT circuit.

This solution is to integrate 1C and other systems through the company's service bus.

ESB is a software solution that works as a single messaging center between information systems and applications. The service bus takes information from the source system in a form in which it is convenient to transfer it to the source, stores it or routes it to a message broker, converting the information into a format for the receiving system, if necessary. Logging and monitoring software solutions that can be integrated into ESB (and in some systems they are already part of the out-of-the-box configuration) make it possible to monitor the delivery of information between systems and quickly identify and fix errors.

A very simplified and conditional scheme for integrating systems via ESB is as follows.

Each system does not send and receive something directly from other systems, but via ESB.

Integration options via ESB

At the same time, ESB cannot be called a “magic pill” for integrating 1C with anything.

First, integrations have yet to be properly designed: to separate domains and information flows. We wrote about this in more detail in previous articles — you can read them, for example, here→ et here→ or go to the general page blog→ and sort articles by tag ESB.

Secondly, to create a connection between 1C and ESB, you will have to develop a separate API for each entity or connect the same OData protocol.

An attentive reader who is already dealing with an IT circuit that includes the 1C product will object that 1C most often exchanges not one or two, but five or even ten data streams with each system. And the actual data exchange scheme looks a little more complicated than a simple web.

When integrating via ESB, we do not suggest “merging” the entire data array through one connector and one stream. Moreover, we highly recommend not doing this, as this way the integration burden increases tenfold and the risk of failures increases proportionally.

The most rational approach, based on our experience in developing integrations, is to distinguish entities and transfer each of them through a separate data stream. Depending on the complexity of storing entity information, you can choose the transmission protocol:

• if the information is essentially stored in one or two tables, it is better to use the JDBC protocol with a direct connection to the database (without API);
• for a more complex data storage structure (for example, if an entity is described simultaneously using several additional directories), you will have to write an API and use the OData, RESTful, SOAP, or GraphQL data transfer protocols.

The logical question is: if you have to write an API anyway, why do we need an artificial add-on in the form of ESB?

Benefits of integrating “1C” through ESB

1. Reducing the load on “1C” and on the end systems

In a point-to-point scheme, one of the systems is necessarily responsible for transferring data and for converting data between the formats in which it is stored. These tasks can be distributed among exchange participants or transferred to one of the systems. In any case, in order to accomplish these tasks, it is necessary to increase the system's code base and increase the load on it.

In an ESB scheme, only the API is an additional element. All data separation, filtering, and conversion activities take place within the ESB layer.

2. Reducing the amount of data transferred

In a scheme with direct integrations, systems are forced to transmit the same information repeatedly. For example, 1C transfers information about orders to WMS, a logistics program, CRM, a seller's reward system, etc. You have to broadcast the same data five, ten, twenty times.

With ESB, instead of sending information about customers or orders ten times to different systems at different frequencies, 1C sends information once with a specified frequency. And already, recipient systems are retrieving this information from storage in the ESB layer by separate integration at the frequency they need.

In KT.team's practice, there was a case where a client had been set up to integrate with three systems. One was responsible for inventory balance (WMS), the second for prices, and the third for product information (PIM). Inventory balances change most often: in order to keep the online store up to date, 1C had to “go” for information about balances every 15 seconds (about 40,000 times a week). But the integration logic was set up in such a way that 1C accessed all three systems every 15 seconds and received 120,000 data packages per week in response. Both 1C and the rest of the systems in the circuit were overloaded.

When the KT.team built integrations via ESB, each system began to receive and send information at the required frequency. WMS still sends inventory information every 15 seconds. The pricing system transmits data once an hour (168 times a week). PIM, on the other hand, has reduced the planned data transfer to once a week, by 40,000 times. As a result, 1C began to collect just over 40,000 data packages instead of 120,000!

3. Accounting for system performance

ESB logic can include the speed of uploading and loading data to final systems, depending on their performance and daily load dynamics. For example, if your CRM can accept 100 transactions per minute during the day and 500 transactions per minute at night, ESB will take this logic into account and will not overload your systems.

4. Timely detection of errors

As a rule, direct integrations do not necessarily include logging and monitoring. The company learns about emerging issues only when they start to affect the business.

A properly constructed ESB layer must include systems for logging, monitoring, and alerting about emerging problems. As soon as there is a problem with delivering messages to any of the systems, engineers or technical support are notified about the incident and its localization.

An example of building integrations between 1C and other systems

Thus, in one of the recent projects for an international e-commerce and retail company, the KT.team set up integrations for several 1C systems: ERP, OMS, UMP, CMS.

We have implemented an integration bus and implemented 13 connectors for seven data streams:

• one connector takes orders from OMS;
• one connector transfers order data to ERP;
• two connectors allow the exchange of order statuses between OMS and ERP;
• one connector transfers order data to the reward calculation service;
• two connectors transfer user data to the reward calculation service;
• four connectors for data on users' currencies and countries;
• two connectors are needed for filtering by warehouse.

DEach entity has a separate connector (or several if required by logic). This allows data to be transferred between systems in a more controlled manner.

CVs

• 1C can be integrated with ESB via the JDBC protocol (direct connection to simple databases) or via an API that operates on OData, RESTful, SOAP, JDBC or GraphQL protocols (if the entity is defined by a complex database with many tables).
• Each entity requires a separate connector and API.
• Integration via ESB allows 1C to transfer each entity once and at a given frequency, regardless of the number of receiving systems.
• Integration via ESB reduces the load on systems, reduces the size of the code base, reduces the total amount of data transferred, takes into account system performance, and weakens system connectivity.

Which ESB to choose for 1C integration

If you have already studied the question, then perhaps the first and logical answer for you would be “1C:Tire”. A product from the same family is really easy to integrate with 1C:ERP Enterprise Management, 1C:Accounting, 1C:Payroll and Personnel Management, etc.

Another Russian data bus, DATAREON, is well adapted to work with 1C products.

But if you have applications from other vendors in your circuit, it makes sense to choose a data bus for a wider range of parameters than just compatibility with 1C. Moreover, practice shows that any ESB can be easily integrated with 1C. For example, we worked with Mule on projects with hundreds of 1C connectors and a high bus load — and we didn't see any problems even during periods of peak load.

When choosing a tire, our customers usually pay attention to the following parameters:

• the presence of an OData connector for 1C (it is extremely rare; non-Russian ESB probably does not have a ready-made connector);
• the presence of errors related to the low-code implementation;
• a method for storing related information about messages;
• the ability to build a microservice exchange architecture with flexible scaling;
• the ability to close the entire area of responsibility of one connector in one job (i.e., a mini-program for working with data streams) or a set of jobs;
• version control;
• stream encryption;
• availability of a visual interface for development;
• asynchronous mode support;
• the presence of a message transformation mechanism and the formats it supports;
• transformation method;
• licensing costs.

If you need to integrate 1C products with other systems and you are not sure which ESB can be used to implement this, write to us in the form.