Operations and Actions

[glibersat]

Hi,

What about using the command pattern for Actions ( http://en.wikipedia.org/wiki/Command\_pattern ) and making Macros ( http://en.wikipedia.org/wiki/Macro_%28computer_science%29 ) for Operations (that are made of Actions) ?

If you're ok with the design, we can prepare the patterns while you're working on the operation themselves. Tell me if you're ok with that.

Cheers

[alcinnz]

Hmmmmm. You'll have to sell me on that. At the moment, it's probably overkill.

In the current design, I don't want operations to be too extensible as that would lead to incompatibility between instances (Federation) when overused. (This may not be a good argument)

[alcinnz]

Few more things to say after considering and looking at the planned built in operations (docs/protocols/wave protocol DTDs). There's enough pure commands (mostly events, Federation, robot, and fetch) and to a lesser degree pure events (mostly blip and return data from the commands) that it would be nice to have the distinction between send and receive.

Also in getting the test system to work, I realized that adding routed events actually clears the roadblock that was in the way and since you can work on more central stuff, it'd be convenient if you did.

This central stuff includes:

• As @atiberghien noticed, the file storage needs rewriting.
• You could, although probably after file, implement you're on storage plugin.
• Implement applyDelta.
• Implement OperationNS.shouldSend and related routines. I've got some unfinished/untested boilerplate here.

I'll be wanting to implement the main protocol, and will probably do it tomorrow.

After these, it'd be time to write the operations. (This architecture seems to be going up fast, yah!)

[glibersat]

About operations & patterns

What I like with the command pattern is that you can :

• Trace the actions, that's much easier for debugging
• Factorize the code (almost every operation will use "retain" for example)
• Write macros that are very easy to read and maintain
• Undo/Redo can be easily implemented, sometimes that's very usefull.

That's really an internal thing, it doesn't mean we encourage wild extensions.

About tasks

I'm investigating the ideas you've given and we'll see with Alban (@atiberghien) how we can do that. We'll give you a heads up as soon as something starts.

And yes, we're all making good progress and that's really pleasant to work with you. Thanks for that :-)

[alcinnz]

I'll write some psuedocode on planned operations and decide from their, but from my analysis so far:

• xPath would decide almost on it's own about whether events should be sent. Since this is the case, I'd like to keep it that way. (the action, or perhaps reverse action, would still build the element).
• Generating the elements would for the most part be simple data moving.

So I don't think that the reverse checks would be much to maintain, but it's all theory at this point.

I'll post my results later.

[alcinnz]

Well, I did some psuedocode for my plans (I'm not sure what the macro would be, and don't think I could be fair there).

As I said already, I don't think the conditionals would take much maintenance, and I don't think what we gain from a macro language would offset what we loose. Besides, we already have a nice, highly flexible language (with a nice XML API, not as flexible). As for undo/redo, for a few reasons (mainly playback and syncing between servers) I store all revisions permanently making that unnecessary.

However, there is much to be gained on your actions. Like debugging and factorizing, but also would make a nice place for making slight changes between client and server on XML.

So my decision is yes on actions, no on macros. Write the actions one way for Python. A few ideas I have for it are:

• Seperate into two modules, encapsulating core and alternative XML views.
• You could add a doc parameter to receive handlers, which loads the doc out of storage for the operation.

[glibersat]

Alright, we're going with a basic command implementation for the core actions then! Once we agree on that, we can build on top the high level views.

[glibersat]

I'm making progress on the core.
Do you have a sample message of what we're going to get when we receive an operation ? Thanks

[alcinnz]

/docs/protocols/Wave Protocol DTDs/

Most of the elements will work as events and/or operations. XMPP roster will also be implemented as operations.

user (empty namespace for custom elements), style, and contrib are purely attributes and don't include any operations or events.

[glibersat]

Hi alcinnz,

You can have a look at my "commandactions" branch ; I'm making good progress on operations and actions.
I now have operations such as move the cursor or insert characters that you can use to make operations, either programmatically or by feeding an XML stream (see tests in core/tests/test_operation.py).

I wrote that assuming operations were only going to be performed on documents, but I'll make that more generic in a few days.

I'd like to have your feedback and/or suggestions about the implementation, so I can improve the design if you have ideas.

Cheers and happy xmas,

Guillaume

[alcinnz]

It looks nice so far & should work nicely. XML based transformation is what I'm developing on the client as a way to render waves and a (looser as data goes more directly in) version should work nicely on the server.

I'll look into it more to give better feedback.

Merry Christmas!

[alcinnz]

O. K., better feedback. The design is cool, but two main issues:

1. (minor) It would probably be more effective in operation code to write the action operations as decorators.

2. I'm concerned about overuse. It wouldn't be effective to factorize code that's clearly used once. Just create very general actions and keep in mind that finalizing would be best when we implement operations.

3. It's looking a bit like deltas which I got rid of before, which makes me uneasy. Just make sure it stays a nice API and not another technical detail, as I'm sure you will.

Well done. Thank you. Merry Christmas.

[glibersat]

Thanks for your feedback!
I'll make sure to take your remarks into account. Could you eleaborate a bit more on 3) ? Is that related to the fact you don't want to send deltas between the client and server? If so, why?

Also, what is the progress on the DTD (document one especially) ? Because how it is currently written doesn't really fit the way wave works, so I was wondering if this was because it's still a big wip or if you had an idea behind that. In the latter case, could you explain that too?

@atiberhien also had a question about the datastores. In the datasource interface, there are only document related things. But, afaik, the datastores also have to persist the hierarchy of waves, the users right, etc. So, do we agree that we need to add new methods to persist these informations, that are not directly related to documents ?

Thanks for your answers & merry xmas :-)

[alcinnz]

Complaint three is basically something that makes me uneasy, but I might be silly for that.

As for why I'm not sending deltas between the client and server is that the operations are simpler in XMPP and the deltas complicate the Wave system and PyOfWave. Of course, these actions as long as they're used correctly can simplify instead.

As for the DTDs, they are closely derived from Google's Simple Data Protocol. However, now the client protocol needs to handle concurrency, so I changed the document DTD to a curser based lock system.

As for the Data Store, I plan to store users under their address and waves, posts, etc. under their URL as per "docs/protocols/Wave URL.txt". The Data Store is designed to store any XML (previously wave document).

I hope this answers all your questions & I hope you had a great christmas and a happy new years.

P.S. I'll attempt to be ready for operations by the new year, so we can start a new part of POW then. You can too if you want. (I didn't uphold it)

[glibersat]

Hi Alcinnz,

First, Best wishes for 2012 :D Hope you had a fun new year party!

Could you explain why you think sendings delta makes things complicated? I think it makes things easier for some complicated parts such as locking. You don't need to do this kind of stuff using Google's OT, and this is why it is successfull. I'm not against having a better protocol (still, I'm concerned about interoperability) but in that case, we have to write a clear document before starting coding. Maybe it's crystal clear in your head, but for us, it is really not easy to understand what you want to do and how ; therefore cluttering our vision of PyOfWave :)

On the client side, why do you have to handle concurrency now ? Isn't that some kind of regression ?

Thanks!

[alcinnz]

Hope you had a good new years too.

There's a few reasons I want to use operations rather than deltas:

• atomicity, making a fully decentralized system (with smaller "failure zones") easier.
• routeability, making it efficient.
• events operations interopability, so Federation doesn't need a separate protocol (O. K., could get same benefit from deltas)
• less protocol abstraction, instead of 4 layers (syntax (XMPP), actions, delta syntax, deltas) I have only the first 2.

Main points are 1 & 4. I know that deltas have been a bit of a stumbling block for some people (I tried to standardize, but it seems we're not ready, I heard this from them).

As for concurrency on the client side, I have ONE XMPP protocol and the one protocol needs to handle concurrency. Besides, the same data has to be specified already (apart from version number) I just moved the target for the data to it's own operations so I can track it.

I'll write a Protocol Architecture Overview document, and if you know some things you need explained, please ask.

[glibersat]

Hey,

Ok, it is now a bit more clear :) Yet, I'm waiting for the document to fully understand your point since I suppose you are going to explain the technical reason of not choosing deltas. I'm sticking to document internals while you write this document.

About the client, we'd like to start too. Someone in our team is also willing to be a tester for this side. What's the state, where should I start?

Thanks!