Thinking about scalability

Building scalable systems consists means designing so that components can easily be added as the load increases. I like to think it also means efficient use of what is already there. Often there will be a query that can take 2 seconds when run one way or, using a different approach be completed in 15 milliseconds.

And so it come to our geo-spacial features: We currently run the Location Butler with a Mysql database. The database in our case is the "glue-without-the-goo" that holds your beacons logs, the the place database and all our user information. We have to do some rather complicated queries to find out who is nearby. Howevery looking at the load with our current nearby search and growing user numbers, this is not a sustainable solution.

So this this week, along with Helge's refactoring of the Java backend, we have been simplifying the infrastructure used for routing around and finding nearby objects.

MySQL as a DB has worked well until now. But our outward spiral search and the nearby searches were not optimal and so we are switching to PostgreSQL. PostgreSQL comes with a really good set of GIS extensions. So we can just as the DB for a set of people nearby using a query like:

SELECT the_geom, user_name
FROM location_points
WHERE Distance(the_geom, 'POINT(100312 102312)') < 5000

Which is very cool because it saves us sucking all the data out of the DB before analysing it. And it means that with our read-only cluster of DBs, we can keep on scaling.