How FixMyStreet Pro copes with a diverse range of different council systems
Full integration: it’s one of FixMyStreet Pro’s main selling points. When a council signs up as a Pro customer, we hook our systems up to theirs. Citizens’ FixMyStreet reports drop directly into the existing workflows, and the council’s updates are fed back to them. Both sides continue to work exactly as they always have, but now they are communicating with one another.
Our dream set-up is when a council uses an Open311 interface, which makes this hookup very straightforward — but it’s pretty rare that things are that simple.
mySociety developer Struan explains the Open311 translation setup we’ve put in place, that keeps everything working smoothly and has the added benefit of keeping our code tidier. While his account is pretty easy for any reader to follow, this will definitely be of most interest to coders and council IT folk.
—
Struan: In FixMyStreet’s early days, all reports were sent to councils by email: this was done with a big script that essentially grabbed a list of any reports that hadn’t been sent, looked up the email address to send them to based on the council and report category, sent the email and marked the report as sent.
This was fine until we started offering proper integration with council systems. Barnet was our first council client, and initially we just added some if Barnet: use other send code bits to the script. But as more councils came on board, this wasn’t really sustainable — you don’t want code full of various exceptions for each council.
Checking out libraries
At that point we rewrote things so that the sending method was stored in the database against the council, and moved the sending code out into a set of libraries with a common interface. This meant that when the sending code got to a report, it could look up how to send it in the database and use the appropriate library to send it. This also means that from our end, changing how a council accepts reports is as easy as selecting a different option in the Send Method drop down in the admin.
One benefit of this setup is that we were able to add a method to pause sending to a council, which is handy if they are doing something like weekend maintenance. Reports get queued in FixMyStreet and we don’t get the many error emails saying can’t send to council X.
This was a fine approach, but as we added more integrations this resulted in more and more code in FixMyStreet which was fundamentally about our commercial operations and not relevant to anyone else installing the open source FixMyStreet codebase for their own country. We needed a way to move this code, and any configuration related to it, out of FixMyStreet.
Speaking Open311
One of the sending libraries that we created sends reports over Open311, which has the advantage that it’s pretty flexible and handles pretty much everything you need to communicate as part of an integration: sending reports, fetching reports, fetching updates, sending updates and fetching report categories. Obviously, it would be lovely if every council could provide us with an Open311 interface, as we could just plug that in directly. Sadly this is not the case.
So, we did the next best thing: we provide our own internal Open311 interface to FixMyStreet which then translates Open311 calls into whatever format the customer’s system requires. This removes the need to add code to FixMyStreet for a new integration, and it also moves lots of the integration specific configuration out of FixMyStreet too.
It works by using the jurisdiction_id that Open311 passes in to load up the correct configuration so most of our council integrations look pretty much the same on the FixMyStreet side.
Inside the Open311 adapter code we do various things to make the actual back end system work like Open311 expects. Some of the calls Open311 makes require multiple calls to an integration to carry out so the adapter does those and aggregates them for the response. We also do things like transforming complicated hierarchies of attributes representing category types into a much simpler list.
The Open311 adapter itself is stateless: it simply translates calls from FixMyStreet and hands back the results. Overall, it makes things much simpler on the FixMyStreet side and moves a load of irrelevant code elsewhere. So far it’s working pretty well with six integrations running through it.
—
Image: rawpixel