Some details
The system was created as a foundation for a project to scraping large amounts of data. Ultimately, it became a framework for building distributed processing systems. The operation of the system is based on the transmission of messages based on the queue between employees, each of whom performs tasks written in JavaScript or Golang. As part of the system, you can build many subsystems and run workers from the GUI level. For
example, building a crawler running on several dozen VPS machines that takes out content from the list of hundreds of thousands of pages takes about an hour.
