Five years ago we as specialists started almost from scratch but were constantly improving our skills so our technology stack was changing.
When we started to work on projects we decided to study and use Node.js platform before it became popular because we found it very perspective. Time has shown that it was a right decision.
By the time we have started working on our search engine Runexto new type of indexaton tools appeared — «headless» browsers, PhantomJS was the first one that we used. But it wasn’t too fast and fault-tolerant and didn’t meet some of our needs. That’s why when we started to develop our web pages tracking system, we chose Scrapy framework + Splash programmed with Python/Lua and YouTrack to track bugs.