Wednesday, January 05, 2011

light up the synchrotron

There was quite a bit of constructive feedback from my blog posting the other day about Synchrotron, my recent pet microproject to make it easy to create and maintain OCS feeds for application addons, and so I thought I'd spend a few minutes sharing some of the feedback and thoughts with you here.

Let's start with the whimsical: Parker Coates suggested this song by Canadian alt-rockers Carbon Dating Service as the theme song for the project. Who knew there'd be a song called "Light Up The Synchrotron"? Awesome.

On a more serious note, some asked as to the security of the service. Well, it's pretty simple: All data sent back in response to requests comes from the PostgreSQL database. All requests are written to be safe against things like SQL injection. If someone managed to find a way to do something untoward with the SQL queries, the scripts that power the public API use a database user with read-only writes to the database with one exception: it can insert rows into the IP access time log table. This (nearly) read-only user can not delete or update anything in the database. PostgreSQL just rocks in how easy it makes to add and manage a layer of security right inside the databas itself.

Other than this database access, the public API scripts do not touch the filesystem at all. So I'm pretty comfortable with it, though I'm still very much interested in people actually looking at the code, however, to ensure everything is indeed in good shape.

Speaking of which, I forgot to mention that there is a TODO file as well as some sketchy documentation in the repository in case you are wanting to get started on something. We also started up the respository for the actual addons where we are starting to collect items.

A few people asked if Synchrotron could replace kde-look.org or kde-files.org and the simple answer is: "no". The reason is that Synchrotron is not meant to allow for uploads and sharing of content by users. It is quite specifically an upstream tool. It's designed to make our lives as upstreams as easy as possible, in fact, but this makes it rather useless as a public file and data sharing hub. In theory it's possible for Synchrotron to be extended to be such a thing, but I have zero personal interest in that. :)

Something I do have interest in is using Synchrotron as a component in some larger projects that have been waiting for attention for a while, such as Plasma Classroom. So I do hope to reclaim the ~2 days spent on this project many times over.

Finally, Josef Spillner pointed out to me that there is the Hotstuff project that he started back in 2004 that strives to do some things similar to Synchrotron. Indeed, this was the project that helped birth the whole Get Hot New Stuff feature that can be seen in many KDE applications today. Sadly, I wasn't aware Hotstuff was still active and a viable candidate, as the last time I'd had a discussion about it I was left with the impression that the Hotstuff server wasn't really being kept up. Hotstuff is written in Perl, which is even less my language than PHP is, and is significantly more complex (and so probably does all kinds of nifty things Synchrotron doesn't); I'd probably still be struggling away by now instead of having something that works if I'd tried to go with Hotstuff. Still, Hotstuff is an interesting bit of software, it's been there for quite a while (nearly 7 years!) and Joseph is still interested in the Hotstuff code. So if you find Hotstuff to be something more like what you need, please get a hold of Joseph! :)

For that matter, if Hotstuff gained the features of Synchrotron and was easy to install/maintain, I'd probably drop Synchrotron for it myself. Necessity, however, is indeed the mother of (re-)invention. ;)

3 comments:

Lukas said...

Interesting thing - a few days ago I started thinking about same web serviced plasmoids problem :)

The main concern is that currently most of them (if not them all) does front end and back end stuff manually and individually.

It's problematic because:
* each plasmoid has to keep up with service API changes.
* Less code reused - like if there were Public transport plasmoid for Berlin, for Vilnius you have to fork/have to start form scratch
* Connection problems - not only in previous post's comments mentioned air port problem, but also getting little DoSS'y -think having many activities + dual screen + An super useful XYZ plasmoid on each of them.
* Auth issues - user has to put his credentials to each plasmoid


**Possible solution**

Create an infrastructure to develop front and back ends as much separate.

On of the ways could be to have a local "proxy" to provide data to plasmoids.
Think about Yahoo YQL but made for KDE.

It should/could include:
* Central credentials management - for one service only one place to set up you accounts
* Naming conventions - all returned data must be "normalized". If it is *title* it is title - not *post_title*, not *articleName*, not even *t10*
* Abstract access - plasmoid could just ask for Most urgent ToDo's, and back end returns (based on accounts user has) from Remember The milk, of Akonadi, or YetAnotherToDosService
* Smart Cache - e.g. weather info is provided for at least a week upfront. If there is no internet just use pre-cache


So plasmoid developers could simply do a http call to http://localhost:99/type=microbolg to get last posts from Twitter or Identi.ca in nice xml/json format
or http://localhost:99/type=microbolg&service=twitter&search='#kde'&since=2h - to get fresher than 2 hours #kde tweets.

Same thing for time tables, RSS, calendars, or even media players (same plasmoid, no matter if Amarok or VLC is currently running)


**To sum up**

* UI experts could do their best about UI's
* More location specific services
* Less headaches about changing API's
* Avoids most connection problems


Hope I managed to put most important aspects here. Comments? :)

Aaron J. Seigo said...

@Lukas: "Create an infrastructure to develop front and back ends as much separate.

On of the ways could be to have a local "proxy" to provide data to plasmoids. "

yes, and it is called Plasma::DataEngine.

it even includes the transparent caching as KDE Platform v 4.6.

the one thing we don't have, which is out of scope for Plasma and really belongs on a user-account-wide basis is "Central credentials management". DataEngine makes this less painful as there is just one place to deal with this for Plasmoids, but it would be much better to move this to a desktop-wide place that DataEngines of different types even could access.

yman said...

And mothers are the necessity of invention.

How do updates work in OCS? I was under the impression that it was unaware of whether or not something is an update.