ElementTree

I've said this before, and I still think it holds true. ElementTree is very nice.

There was a recent discussion about it on comp.text.xml, which got the attention of Fredrik Lundth

I do much apreciate ElementTree. Of course using a non-DOM API to process xml conviniently is heresy, and will call the gods crushing down on us.

I find the proponents to a strict and true die hard one Document Model API hillarious in their self imposed blindness towards pragmatism.

But then, it's not my time I'm wasting on cumbersome and broken API's, or is it? Of course the Net-dammage in the form of popular implementations of DOM can't be taken into account lightly...

PyPy 0.6 out

Alright, now this one gets me quite excited. Just 5 minutes ago I argued with a friend of mine, that PyPy is in fact going forward, and he argued that they'll not have a release anytime < 10 years.

However, there is a release 0.6 now. From the tutorials and explanations, there's plenty a mention that this is higly experimental. I can spot type inference there, as well as translation to machine code ( or at least something very very close ).

This is exiting because once PyPy, executed on the CPython can translate to machine code, it could compile itself to a running programm. One which in turn can again compile future versions of itself and so forth.

Not only does it mean that python gets very flexible with this, it also means that integration with native code, runtime speed and a ton of other things get very fancy now.

Onwards to world Domination with PyPy ;),
Kudos and good Karma for the group around PyPy, you do an awesome job!

RPC and protocol bloat.

While implementing rpc stuff, I noticed that the protocol bloats. Here's an example.

remote.text( 's' ) becomes ('text', 0, ('s',)) which becomes some 40 or so bytes once marshalled. A byte id and a single byte for the letter would be 2 bytes in contrast.
The problem about ('text', 0, ('s',)) is that it has more information. It carries the method to be invoked remotely, what type of command it is ( i.e. a call and not an error message ), and the argument-list for the remote method in an easely digestible form.

I realize I could optimize somewhat with a fancy marshaller, that negotiates formats with foreign marshallers, like templates or identifier lookup lists.

But then let's say each tiny message translates to abou 50 byte. a user can only type around 2 baud, so the generated netflow would be around 100baud/user. you can support at least 128 users on a 24kB modem line then ( assuming they do not do worse things then typing and pasting short text ).

Is optimizing at this point really worth it?

I guess I should just encapsulate properly, and later if I really want I can still try to optimize ('text', 0, ('s',)) structures to a few bytes.

RPC, Nodes'n stuff

I've been hacking away at an rpc mechanism. Fully knowing that I ignored proper asynchronous dispatching, just a very thin layer over messaging. However, during the process I noticed that it would be rather intersting to leave the client/server pattern, and instead provide a generic object that is a node which can both accept and build connections. I'm not really sure how a node will find out in a nice way where to connect to, but I think it's interesting none the less.

Networking

I've been on it again. I don't know why precisely I torture myself with network programming. There's a pleathora of toolkits for precisely that readily avaiable.
Generally I do scratch-writes of existing libraries only 1/4 until I grasp the concept fully and convert to an existing library.

The things I've had direct exposure to where asyncore, medusa, twisted, xmlrpc, tcpserver and of course socket. I think there's far more, but these are the ones that notably stick around my internal memory so they might deserve the mention, honorary at least.

I can' really reason that any of those above would be insufficient or somehow lacking, they just do not really apeal to me.

So, I'm on the fourth ( or was it the fifth ) rewrite of my network library. Somewhere at the second rewrite I decided asynchronous can't be done properly, and sacrified it for threads. Meanwhile having aquired more network-karma, I actually reversed my opinion, and think asynchronous can in fact be done properly, and it will be a big merrit in the end.

Of course I reinvent all sorts of wheels in the process, including but not limited to rpc mechanisms, message transport protocols and asynchronous messaging.

I still don't know really why I do it. But there's only two possible explanations. Either all existing stuff sucks beyond comic relief, or I've still not grasped the fundamentals of network programming, even after years of exposure.

I fear the latter is far more likely, but duh... can that really be?

Scheduling

It was monday afternoon. A friend of mine aproached me with a seemingly simple problem. The problem is as follows:

-There's a number of tasks, that have to be repetetly called each second. The amount of each task's repetition is fixed, and has to be met.
-Some tasks have a dependency, and thus execution order for those matters.
-One task may be used as a fill-up, when there's time left, or to drop if there is to little time.
-All tasks should be evenly distributed, in order to make the system apear as fluent as possible.

( note, these requirements are already a bogged-down version, that tries to eliminate a ton of complexities, or so we thought )

As the day progressed, and we talked for hours and wrote code and tested, it turned out this isn't a simple problem, but a hard one. In fact, we ended up with something that performed bad, and on top of that couldn't fill-up properly.

I told my friend up-front that I think that this is not one of the simple problems, vaguely remembering some school lessons about operating systems, and process scheduling. However, we tried to adjust requirements to a form that made it all implementable, seemingly, just that we ended up having still difficulties getting it right.

It was kinda Bummer, I feel I need to ponder this problem longer, and understand more of the theory to get it precisely right. I hope I can come up with something vastly better when I get to the third or fourth rewrite.

Patching

Once in a while a pc-game hits the market that needs patching badly from the get go. In the best of cases the patches come fast, eventually they come before the game is released. The producers of such games tend to fall short in favor of critical customers, while the press joins the chorus. I'm not sure if this is the right kind of bad publicity that's as good as good one.

On any account, I'm baffled by the producers of said pc-games, because they could easely avoid the trouble. Instead of having customers find, download and install a patch ( by which time even the very simple minded start to get clues ), they could just incorporate automatic-patching.

Without going in to technobabble, just give the user a cute little box, offering him to update/restart the game when an update is avaiable ( if you feel brave you could also try live-patching, warning though, not for the faint of heart ).

Chances are your user will just klick yes without reading, that's about 70% people less that are even noticing you're patching the game short after you released it. And the remaining 30% will be much less enraged because the actual efford they had to spend to get the patch migitates their feel to have been screwed by you. I even think that automatic-patching is percieved as quite normal, whereas hand downloadable patches are precieved as literally patchwork philosophy ( altough technically it's both the same ).

Make Wrong Code Not Compile/Inteprete

I like Joels latest article about making Wrong code look wrong. The whole shebang about making relationships as visible as possible and stuff.

I still do not like Hungarian notation and I do not agree with his assertion of the best solution.

The first reason is that it is dependent on a reader to tell an error. Everbody likes errors, and I like them best when they do not interprete/compile.

Which leads me to my second reason. The hungarian notations solves problems that can be adressed by other means as well. For instance type-subclassing. However this hasn’t deserved a mention in Joels post, which I find somewhat dissapointing.

Normally code is litered with dependencies, intricate semantics and stuff that makes it harder then to write blog entries. I’m pretty glad the less there is to worry about and the more is automatically checked. Hungarian notation introduces another layer of semantics on top of the existing complexities, I find it distracting.

The next reason I dislike Hungarian notation is that it forces you either to use acronyms in names, or waste rediculus amounts of name-estate.

I dislike acronyms in names because it is an additional inderection, with which I’m forced to deal in order not to make errors. Indirection because extensive use of acronyms ( as in hungarian notation ) leads to the code beeing litered with acronyms, and each of it means something that’s supposed to tell you something, that you have to remember in order to find out what it means, instead of telling this to you up front.

Obviously screen estate is limited, and so is name-estate. I like to spend name-estate on reasonably describtive multi-word names ( dislike commenting the obvious really, if I can avoid it with good naming ).
Hungarian notation can’t do that for me. But it can harm my ability to form good names, by introducing more bytes stacked on top of my names, and thus taking away valuable name-estate from me.

Firewall admins, Corporate Logic

Dear reader ( if any ) I am forced to post this comment in IExploder, and do the rest of my surfing using this godawfull insecure browser. Why you might ask? The oh-so-wise(tm) firewall admins at our company obviously decided that two critical security holes in Firefox are enough to filter out any traffic going on using the UserAgent: Mozilla*.

Ahyes, this makes sense(tm) to me too, whatever happend to the dozens of unpatched critical security holes in IExploder, and it is preinstalled on our machines, traffic using UserAgent: IExploder/Malware Magnet is still alowed...

I think it's time to sigh in dispair(tm)

MetaGer - Does software move forwards?

This is an interesting article

Recap:
Pupil found paper of head of a research facility, about the next generation search engine. At the time of the writing it was not feasible mainly due to network bandwith constraints. Pupil cleverly implemented the idea salvaging his existing broadband account, and now everybody's abuzz.

What struck me about this is that said pupil not only implemented the whole thing from scratch, he also found a way to cleverly implement the new features in a way that is feasible today. And all he needed was a describtion of the idea.

Amazing feat! It will not surprise you much that the pupil used python for this. In face of this, I suspect that under the right conditions python's not only more productive, but by orders of magnitude so.

Texture Packing

I toyed a round a bit with texture-packing. The idea is to pack multiple arbitary sized textures on a bigger one. In the context of a sprite engine, this makes sense. Today when you write games, whatever you do, you use the exisitng 3d hardware, even for 2d stuff. The thing you want to avoid when rendering with super-fast hardware is shuffling data around between card and ram. Thus it's a sensible aproach to load all textures needed for a batch of work once, and then have primitives use them.

Anyway, here's a pretty pic :D

( p.s. of course this is all written in python, source follow later )