it's just me...

Skewed distribution

2011-03-06T13:35:00.004+01:00

Distributing items (IP packets, phone calls, http requests) randomly onto a set of machines (say, four) is easy (in C here):


int slot = random_nonnegative () % 4;

The number space returned by the random number generator is big enough that the items are essentially evenly distributed even if the number of slots wasn't a divider of the size of the random number space.

I want a skewed distribution, however. The slots shall be loaded unevenly. Thus:


int slot = random_nonnegative () % 4;
slot = random_nonnegative () % (slot + 1);

That is, if slot is 3 after the first line, then we have an even change to get to any of the slots in the second line. If it is 0 we can only end up there. In essence there are four ways to get to slot 0, three to slot 1, two to slot 2, and one to slot 3. Zeroth-order intuition would thus say that four out of ten (1+2+3+4) items would end up in slot zero.

Except that the experimental results don't match. And for a reason. For slot being 3 in the first line, a fourth of the item go to each slot; for slot being 2, a third goes to each of the three first slots, and so on. Thus slot 3 receives a quarter of a quarter, or 1/16th of all items (and not one tenth); slot 2 receives a quarter of a quarter likewise, plus a third of a quarter via slot being 2 on first line; slot 0 finally gets 1/16+1/12+1/8+1/4, which is 3/48+4/48+6/48+12/84, which is 25/48, or about 52%. Which is a bit more than the 40% of our first guess.

(PS:


char *target = targetarray [
     random_nonneg () % (
           random_nonneg () % (sizeof (targetarray) /
                               sizeof (targetarray [0]))
            + 1)];

for the golfing old-style C hacker.)

svn needs IDE integration

2009-12-23T21:35:00.003+01:00

It's funny; all the world craves for svn integration for their IDE and doesn't want git because of lack of such integration. On the other hand I am quite happy without the integration, but then I was raised on a command line.

And then I noticed that, for the usual IDEs (java dev, that is), you need IDE integration for svn for a very simple reason: refactoring. If you rename or repackage a class, the source file changes location, and svn needs to record that as a file rename, and you need to tell it so. Now, if the IDE has moved the file, you can't do again with svn, and that would be the only way to make svn to know the move. (You could stop the IE, move the file back and then with svn move it forth again, but...shudder.)

Thus the IDE must be able to talk directly with svn. It's not just a question of avoiding a suboptimal command line, it's a necessity. With git, on the other hand, there is no such need as it does not directly track renames anyway and is much smarter about them and merging.

svn: more bad

2009-11-15T12:19:00.004+01:00

Ranting about tortoisesvn and it committing new files with CR/LF in them, I finally found that there is apparently no way to make svn behave the right way (in our sense) of doing svn:eol-style=native automatically. You need to do a setup like this. Yet another misdesign. (And if you work on two projects where these defaults differ, then you are out of luck.)

By the way, somehow nobody did think that a by-line diff of svn:externals would be a good thing. We usually have one directory with six or so externals, and when you change one of them svn diff just shows the whole property text as changed, and not individual lines. Tortoisesvn is smarter here (as in other places), reinforcing my claim that with svn you need GUI support because the command line interface isn't exactly pretty.

svn: BAD

2009-11-08T22:59:00.003+01:00

svn is just broken.

Google for git commit. First link points to the current documentation. Dito for svn commit. First hits point to outdated versions of the manual (usually 1.0, maximum 1.5).

Checkout a module. Unfortunately, the URL to check out usually ends on trunk so the default sandbox directory name isn't really helpful. Likewise, it took 'til 1.6 that they found out that typing the repository URL in many commands is a waste, and allowed ^/repo-relative paths not only in svn:externals but also in commands.

Just for kicks try an svn ls http://... operation over GPRS. Will take on the order of half a minute because GPRS round trip time is about a second, and svn ls does a good dozen http requests to the server, all serialized, but not reusing the connection. It is actually a good idea to prefix commands with ssh elsewhere; the ssh login is faster, and the svn command is then done on fast links. (Doesn't work for commands involving a sandbox, of course.)

svn import can only import a given tree into a new location within the svn repo; it can't do updates, like they are needed to import newer versions of vendor software and keep the file identity that will be needed to keep future merges smooth. There is a script in contrib that can do that, but it never got integrated into the import command proper.

Likewise, don't even expect a command that can create an archive (tar/zip) from the versioned files in the current directory (git archive), or that can delete all unversioned files in the sandbox (git clean). Or that tells you a human-readable name for the current revision based on the last tag (git describe).

And, of course, there is the big thing: the hyped merge support of svn 1.5 is broken by design. Isn't fixed in 1.6, and probably can't be fixed without changing the repository format. (I suspect the whole mergeinfo property stuff is just wrong.) In the meanwhile, and in a much shorter timespan than from the announcement of svm merge support to its eventual functional delivery, git appeared, and just did it right.

I don't like that. We've been waiting for svn 1.5 because of the merge support, and then started to switch. At the same time git appeared on my radar; I could use git cvsimport and git filter-branch to do our conversion from cvs to svn, in a way that allows to pull future changes from cvs to svn as well, and without manual intervention. Unfortunately I lost all liking for svn in the process, because git turned out to be just much better tooled, capable, and flexible. And faster in its development, its version number is bound to pass that of svn soon.

The awesomeness of no rename

2009-10-22T22:43:00.002+02:00

git does not store renames. All it does is store each version of the whole tree of the managed project. The diff and merge tools are those that look at the trees and file contents and notice that a file has been renamed.

For example, I had a project where a program was converted to C++, piecewise. One file was originally foo.c, then foo.cpp was created by copying and fixing the original C source. foo.c was not removed until later, so for a few commits both existed simultaneously. Indeed this was in CVS, and I just imported the stuff to git, to work on newer stuff there.

Now it was time to merge the line originating in CVS into the work branch. git merge just looked at the changes that needed to be merged all at once, and saw that foo.c was gone and foo.cpp was new over that stretch, and that they were 95% similar, so it assumed this was a rename, and properly merged the rename into the work branch.

That is the power of not doing or saving renames.

git and cvs: Coexistence

2009-09-28T21:26:00.004+02:00

There is a relatively trivial way to use git on trees managed by cvs which I use often to carry around cvs projects for new work and enjoy the distributed nature (which comes quite handy when on a train). Recipe: Check out the target tree with cvs, go into the root, git init, and cvs-files | xargs git add. One git commit -m initial and you are ready to go.

Now you can clone around as you like, and do cvs updates in the gitted sandbox, and commit those into git, and back. Only thing is that on the way back to cvs you manually need to track file additions/deletions.

The script cvs-files looks like:


#!/bin/sh
find * -name CVS -type d | while read cdir
do
  dir=`dirname $cdir`
  if test X$dir = X. ; then
    dir=""
  else
    dir="$dir/"
  fi
  sed -ne 's:^/\([^/][^/]*\)/.*$:\1:p' /dev/null ${dir}CVS/Entries* | \
    sort -u | while read name
  do
    if test -r $dir$name; then
      echo $dir$name
    fi
  done
done

It just goes through the CVS/Entries files to find out what's under CVS control to initially put those into git.

There isn't enough coffee in the universe

2009-04-07T23:22:00.003+02:00

EVS is me going "Screw it, I'm not playing any more" and writing a
system that can talk to anything, given enough time and coffee.

And this mail came after a loooong day filled with nothing actually, unless you count trying in vain to get some GeForce running with openSuSE as work. (Which would have been fun if the driver would not only support portrait mode but also draw correctly and use the panel's natural resolution. eclipse look quite good on a portrait screen.)

Anyway, how can one get the idea that all version control systems are basically created equal, and that that is enough to write an universal server that can talk each protocol equally and fully? Even with cvs you can trivially create a repository than can't be converted to, say, git, and vice versa create a repository in git that can't be represented in cvs. In the latter case the revisions would all be there, but the merge information would be missing.

If you take out all the edge cases, that is, sufficiently many of them, then it may be made to work. But neither would many existing repositories match those requirement, nor would the resulting functionality be very interesting.

MacOS surprise: ssh-agent

2009-02-25T20:46:00.002+01:00

I'm new on my macbook. Anyway, since leopard ssh asks for passphrases using a dialog instead of on the command line. This is not necessarily a bad idea, for example a git push from within git gui has no good place to ask.

On the other hand, using ssh-agent for that is even more convenient, and since this dialog allows you to store the password on the keyring I assumed that that is the preferred way and no one would bother to start an ssh agent under the whole of the leopard gui session. Turn out I was wrong. And my workaround of using xterms started from another xterm with an agent in there was quite unnecessary.

ant: BAD

2009-01-01T13:46:00.002+01:00

Who doesn't know history is condemned to repeat it. Or not even that, as ant managed. In my option, ant started with a bad idea and then made it worse. In other words, it is Broken As Designed.

The quotes are form the aforementioned page, section 'Apache Ant'.

Apache Ant is a Java-based build tool. In theory, it is kind of like Make, but without Make's wrinkles.

Yeah, the wrinkles have been replaced by pointy brackets, and all the deep problems haven't even been addressed at all.

Why another build tool when there is already make, gnumake, nmake, jam, and others? Because all those tools have limitations that Ant's original author couldn't live with when developing software across multiple platforms.

Well, at least I don't want to live with ant. As little as possible. Granted, make isn't usable for building java projects either, but then neither is ant, without working around the buggy javac task.

Make-like tools are inherently shell-based -- they evaluate a set of dependencies, then execute commands not unlike what you would issue in a shell.

That's the point! Why invent another scripting language when you can have one for free? Instead ant goes along inventing a new scripting language (which may not even be turing complete), and it takes them years to come to a point where ant can at least do what the good old shell always could.

And still, to do a simple grep Whatever | sed -e s/XX/YY/ you need to write java programs, compile them, feed the jars to ant, and after a few hour get it actually working. Not to menting the fact that those classes can't easily be part of your project because you need them before starting ant. (Ok, that's not actually true, but in an ugly way.)

This means that you can easily extend these tools by using or writing any program for the OS that you are working on. However, this also means that you limit yourself to the OS, or at least the OS type such as Unix, that you are working on.

Gladly so. Nowadays you can easily get VMs oder cygwin if you should really need to work elsewhere.

At least it's much better than making everything complicated, everywhere. The average trivial build.xml contains about 60 lines, of which three do actually vary with the project, and the rest is always the same. Don't repeat yourself, huh? At least we hope it's the same, and there aren't any copy&paste bugs lurking.

Makefiles are inherently evil as well. Anybody who has worked on them for any time has run into the dreaded tab problem. "Is my command not executing because I have a space in front of my tab!!!" said the original author of Ant way too many times.

The original author of ant obviously had the wrong editor, or was missing a little script to check for that. I've written my share of makefiles, and I can't remember ever running into that problem.

If only the original author of ant ran into the real problems that make poses; then ant wouldn't have been such a terrible misdesign.

Ant is different. Instead of a model where it is extended with shell-based commands, Ant is extended using Java classes. Instead of writing shell commands, the configuration files are XML-based, calling out a target tree where various tasks get executed. Each task is run by an object that implements a particular Task interface.

So, instead of writing little one-liners I need to write code in a programming language that itself is known as verbose, write additional build targets to get those compiled, and finally knit it all together in XML, yet another source of bloat to the whole end. (And for the nitpickers, ant uses two extra syntactic elements: comma-separated lists and property value replacements. Guess why they don't do that as XML elements.) ant has a serious Dr. No syndrome.

Granted, this removes some of the expressive power that is inherent by being able to construct a shell command such as `find . -name foo -exec rm {}`, but it gives you the ability to be cross platform -- to work anywhere and everywhere.

...and making that a lot of work. Ant really does not fall into the category 'make the easy things easy and the hard things possible'.

And hey, if you really need to execute a shell command, Ant has an <exec> task that allows different commands to be executed based on the OS that it is executing on.

Now that't a cop-out. If our uber-tool happens to not support what we want we can still go platform-specific. Thanks; due to general disgust with XML I wrote a little shell script (of all things) that does everything I need to do in the projects I need to manage, including compiling and collecting used libraries of my own. /bin/sh isn't exactly the best way to do, but the whole thing is just five times larger than the (broken) build.xml I got for those projects (and shorter than this blog entry), and the actual build script for one of those condenses to


  #!/bin/sh
  . "`dirname "$0"`"/../proj-a/tools/buildtool.sh
  depdir ../proj-a
  javacomp
  mkjar proj-b

and is also a shell script.

Now what?

Ok, more details on how ant is misdesigned. The javac task is broken:
It only compiles java files that have no corresponding class file or
whose class file is older than the java file. It does not erase class
files for which there is no longer a source file, nor does it
recompile dependencies between the classes. Especially the latter
makes for interesting bugs. Workaround: Always clean before compiling,
either manually or by the 'dependencies'.

And these 'dependencies'. They aren't, really. The individual tasks sometimes execute conditionally depending on whether the destination is newer than the source, but you need to state dependencies explicitly, no matter whether ant could deduce the dependency itself. For example, when a javac task produces what a jar task consumes ant won't make them dependent automatically. Otherwise, when you put a target into the dependencies list of another, it is executed, no matter what. As opposed to the following make fragment


  prog : main.c gen.c
      cc -o prog main.c gen.c
  tool : tool.c
      cc -o tool tool.c
  gen.c : tool gen.in
      ./tool gen.c

which tells that gen.c needs to be generated by tool which in turn needs to be compiled from tool.c. The point here is that when you call make, the tool won't be recompiled unless you modified its source, and gen.c won't be generated unless you modified either the generator or its input file.

Ant stays blissfully unaware of any of this dependency management, and thusly degenerates to the fixed execution of a number of scripts (called targets) with a number of commands (called tasks) each. If you want your task not to do work when none is needed, don't expect support from ant. Ant is not really a build tool, it is a simple script executor, however their creators talk about declarative operations and the ant way which seems pretty long-winded to me.

The 'declarative way' does not keep people from relying on the fact that the dependencies of a target are executed in the order they are specified. A dependencies="clean, compile" to work around the javac problem is all but uncommon, and clearly will break when ant decides to run the dependencies in inverse oerder.

To be fair, the dependency problem isn't exactly trivial, especially without support from the actual compiler. make doesn't do a good job for java, either. But on the other hand side we'd expect an industrial-strength build system to have invested not just a little thought?

Then ant completely ignored the lesson of the X windowing system. Those guys actually improved on make by using the C preprocessor. Their Imakefile system inspired another (proprietary) system that could just say


  CProgramFromSources (prog) {
    CSource (gen)
    CSource (main)
  }
  CProgram (tool)
  gen.c : tool gen.in
      ./tool gen.c

with one important difference: The macros expand so that not only make prog does the expected thing, but also that make clean removed all the temporaries, except for gen.c which was done with a plain make rule. We need to add


  clean ::
       rm -f gen.c

to make that work, too. This also shows another overlooked make feature: You can actually combine a target from multiple separate commands and dependencies. It's just not possible to have multiple clean targets in ant, making it even more error-prone to do the right cleanout.

Ant simply aims too low by one or two levels of abstraction.

Then what?

Unfortunately ant has gotten a lot of traction in the java community. Proves again that you need to be the first, not the best. Compounded by the fact that most programmers don't care about the build system any more than needed to make it apparently work. And everything and the kitchen sink is available as an ant task, so it's not just programmers to turn around.

And I'm not exactly in a position to get a lot of traction for a change. The most promising way is to fight the system from within; perhaps by actually having some preprocessor (again) generating a tmp/build.xml to then be included.

The most depressing thing is that ant will make the majority of people think that this is the state of the art in build system design. Far from it.

Factoring out helper classes

2008-12-31T21:22:00.001+01:00

The job is to get a source and a destination path for a java compile. Because it's mostly the same, we want defaults for source and class directory. Because we occasionally have more than one separate hierarchy to compile, we want to provide a base path on which the defaults are applied. Thus this is the wet (aka non-dry) code:


    AttrList al = new AttrList (l.getParam ());

    String base = al.pull ("dir");
    if (base == null) base = "";
    else base += "/";

    String src = al.pull ("src");
    if (src == null) src = "src";
    src = base + src;

    String cls = al.pull ("classes");
    if (cls == null) cls = "classes";
    cls = base + cls;

We first get the base path and fiddle it a bit, especially when not given. Then we obtain source and destination, set it to default when not there, and prepend the base (which we fiddled so we can always do so even when it does not change the path).

The repetition looks ugly. So: Factor it out. We can't purely do it with a helper function, so we need a class (this is java, everything needs to be done in a class):


class Basifier {
  String base;
  public Basifier (String param) {
    if (param == null) base = "";
    else base = param + "/";
  }
  public String basify (String arg, String def) {
    return base + (arg != null ? arg : def)
  }

and use it:


    AttrList al = new AttrList (l.getParam ());
    final Basifier base = new Basifier (al.pull ("dir"));
    final String src = base.basify (al.pull ("src"), "src");
    final String cls = base.basify (al.pull ("classes"), "classes");

Not too bad, except that we need to define a helper class and instantiate it for each invocation of ours; and we note that the useful lifetime of the object is tied exactly to the lifetime of the invocation of our funktion. In my optinion the latter is a sign that we are doing something wrong: If two things always have the same lifetime, they should really be parts of one thing.

If java had local functions and closures, we could spare the separate class:


    String base = al.pull ("dir");
    if (base == null) base = "";
    else base += "/";

    public String basify (String arg, String def) {
      return base + (arg != null ? arg : def)
    }

    final String src = basify (al.pull ("src"), "src");
    final String cls = basify (al.pull ("classes"), "classes");

We save the need for the extra object, and the need to reference it. Arguably the helper class has the advantage that the basification is encapsulated there, but it is not just that but also the default value handling for the other parameters that is done in there. Besides, the basification may also be usable in other locations.

One interesting sidepoint is that the basifier is almost, but not quite, implementable as an anonmous class:

    
    final Whatever base = new Object () {
      String base;
      {
         base = al.pull ("dir");
         if (base == null) base = "";
         else base += "/";
      }
      public String basify (String arg, String def) {
        return base + (arg != null ? arg : def)
      }
    };
    final String src = base.basify (al.pull ("src"), "src");
    final String cls = base.basify (al.pull ("classes"), "classes");

The only thing that does not work is that we cannot name the type of the anonymous class for declaring base, and we need that type to be able to invoke basify on that object. In scala that actually works by type inference; the variable is just declared as a variable, and the type is assumed to be the type of the initializing expression.

Is the class Basify something that is worthy of a separate existence, or is just a little bit of code that gets factored out, and because of the needed scope of the data it works with, it is forced to become a separate class. It may also be done as a static inner class; only the option of a local function is out.

Basically in the latter case the function invocation serves as a kind of object; we could view the local function as a member function of the invocation. If that is so, we might also view the function invocation as some kind of a class, and then we might actually make it to extend another class (or implement an interface):


    public void methodAsObject (String p, String q) extends Basifier {
      super (p);
      final String here = basify (q, "def");
      ..
    }

It only does not work because we don't have the constructor parameter at that time.

Over the fence

By the way, other languages just do it differently. In C++ you decide whether a local variable is just a reference to an object


  Basifier *base = new Basifier (param);

or whether you actually want the object itself to be a local variable:


  Basifier base (param);

In the latter case the lifetime of the object is directly tied to the function invocation. This can't work in safe garbage-collected languages because you can't remove the object before you know there are no more pointers to it, and this is incompatible with with stack allocation.

Oops

So, this was planned to be a rant against excessive objectification and turns out to motivate me to actually use the basifier class. Will eventually show up here; look for "javac".

svn: BAD

2008-12-30T09:58:00.003+01:00

Executive summary: Read about svn merge --reintegrate and weep.

We waited long for subversion 1.5 to come out. It was hyped to have proper merge support and thus be a real step forward for cvs users. Actually we were using cvsnt which already had some merge support until they broke part of it, and we kept using a patched version.

Anyway, 1.5 finally came out this summer, and since we did not only want to keep having merge support but the history-preserving renaming of subversion as well, the step was done. (We were not migrating but only using svn for new projects.)

Then I started reading the updated svnbook, and wondered: Why on earth is there an option --reintegrate? It turns out that you can only open a feature branch and merge repeatedly into it, to keep it in sync with new stuff on the trunk. But you can only once merge back into the trunk or wherever you came, and then the feature branch must be killed because further merging won't work.
And then it turns out that svn 1.5 was released with this half-working merge support on purpose because it was overdue already and they managed to represent the merge info in the repository in a way that makes a correct implementation impossible, and thusly, it won't happen before 1.6 or so.

And in nearly the same time frame, another star appeared and ran circles around svn in terms of ease of use (just count the number of times you need to type absolute urls in svn command), possible workflows, breadth of utilites. And it's a nice svn client, too.

git svn without root

2008-12-23T16:19:00.003+01:00

git svn is my favorite way of working with svn. Uses less space than a native svn sandbox, still contains the complete bloody history, and doesn't fool grep-find with the pristine copies.

Installing both the right way isn't quite easy. For git you just need the standard install, but you need to install svn including its perl bindings. And you can control the installation location of svn with the usual --prefix=$HOME/local to configure, but this does not change the place where make install-swig-pl wants to place the shared libraries and other perl binding code. You can only change that by doing some steps manually:


tar xzf subversion-deps-1.5.4.tar.gz 
cd subversion-1.5.4
CFLAGS=-fPIC ./configure --prefix=$HOME/local
make
make install
make swig-pl-lib
make install-swig-pl-lib
cd subversion/bindings/swig/perl/native
perl Makefile.PL PREFIX=$HOME/local
make install

Basic point: You need to invoke the perl make separately, and tell it where to install. Seems not to be so easy to include an option for that into the toplevel configure?

Also note the need to manually say CFLAGS=-fPIC even though the target system (Novell SuSE) isn't exactly unusual, and configure should figure this out.

Anyway, the perl library path (PERL5LIB) seems to already include $HOME/local, so after above commands, git svn works. If you use a different location you need to fix PERL5LIB.

git on svn

2008-12-20T21:11:00.004+01:00

git can push onto webdav-enabled http servers.

svn is a webdav-enabled http server.

Unfortunately, it does not quite work. Cloning via http://svnhost/whatever.git/ works, but I can't push back there. It says error: Error: no DAV locking support, and I tend to believe that. svn can lock, but apparently not over http and svn urls, and equally apparently locking is not enabled on our server. And I can't change that, and setting up my own svn server just to test that is out of the question.

Not to mention the usefulness of this exercise. Operating a git repository versioned in svn is...strange. It just would enable one to really use git and still claim to work with svn.

Under attack

2008-12-16T11:25:00.003+01:00

I have a DSL link home, and by now I operate a regular unix system for the router, and it's the first system I have with an internet-facing sshd. Now, every once in a while some bot comes along and seems to try out a lot of account/password combinations, not withstanding the fact that I only enabled public key authentication.

Idea: Whenever massive login attempts are detected, just reflect further incoming connection attempts from the same address back to that address. Thus the bot just attacks its own machine.

Question: How to do this without hacking the sshd itself? Possibly temporary static nat rules?

git usage

2008-11-08T12:28:00.003+01:00

cvs was easy to take on, even though I didn't know any version control system beforehand.

git was harder, even though I did know some systems by then.

The difference: cvs has basically one way to work with it, while with git you are much freeer in how you want to work. With freedom comes the problem of choice and the need of experience. For example git practically doesn't let you destroy any history, but sometimes it can be tough to find out how to recover from a particular mistake.

Also in the mix: When I started cvs, I had nothing. When I started with git, I had quite some history to import and to deal with.

As long as you don't do branches (and with plain cvs, you better shouldn't), cvs is just a way of sharing a common tree without risking accidential overwrites. With git there is a wealth of things you can do, and of those you can do nearly right. I don't think that I am yet in a position to foist git over a bunch of developers and to give them enough training so that they can get along without big mishaps or frustration. Especially not for windows guys.

So mostly I still use git-svn, live with the company decision to use svn, and sometimes use git hackery for special effects that nobody else needs to know about that git was even involved.

Comment notification

2008-11-07T13:39:00.002+01:00

Blogger bites too. I just noticed that there are some comments here, and I remember turning on email notifications for comments, and relied on these.

Consequence: Only today did I even notice the existence of any comments on this blog, for which I do want to thank you, dear readers!

(Note to self: Rework the post labels.)

Priorities and signs

2008-11-07T12:54:00.003+01:00

This was almost going to be titled 'signs are hard'. I hat a problem that a simple timer queue implementation did not quite work. Problem: Some timers seemed to vanish from the java.util.PriorityQueue, or were in the wrong order. I tried, for lack of other ideas to change the sign of the compare method, to see if I got that wrong. Letting the handler thread wait for and pick the latest timer is not a good idea. But alas, that wasn't it, and there goes the title.

Google ("java PriorityQueue remove"): Somewhat unexpectedly, the first result is not the documentation of the queue but rather exactly the report for the bug that is biting me: remove does remove the first element it finds that compares equal to its argument, not that exact object: Voila: wrong element removed; that timer never fires, I'm not getting #ticks.

Good. So now I know what's wrong and can finally implement a strategy to do fewer removes in the first place. Many timers here serve as a timeout and leave the queue not at the front but by remove. And the timeout is constant. So if a timer is set to a later time I can just leave it in the queue and reposition it when it finally appears up front.

Also there is a decision to be made whether I leave the workaround in. It's fixed somewhere in 1.6 (b51?), and I can't rely on the JVM always being that new.

cvs to svn via git

2008-09-24T19:06:00.006+02:00

It actually works. The problem: I wanted to take the history of a partial cvs repository into svn. Directly using the tools was out for two points: First, one needs access to the svn repository itself (at least I think so and didn't bother to check because of) second I wanted to patch the paths of the java packages contained therein. Apparently eclipse doesn't quite want to deal with company_name.com, so it needs to converted to companyname.com even though the company only owns the former domain.

Anyway, I did not figure out yet how to use git-cvsimport and git-svn both on the same repository sensibly; and I think git-svn rewriting of the commits doesn't quite make that feasible. But the problem already starts with getting both histories to have a common ancestor. Unfortunately git does not have a null commit as the universal base.

Ok, final approach: Create project base directories (the ttb) in svn, do an git svn clone on that. Separately, use git-cvsimport to get the history from cvs into git. (Caveat: The approach does only handle a single linear history well.) Use git format-patch --root commit to get a series of patches, run those through sed -e s:company_name.com:companyname.com:g (which patches directory names as well as imports and package declarations). Then apply (git-am) the resulting commits in the git-svn repository, and git-svn dcommit them. Done.

Except that my link to the svn repository isn't exacly fast at the moment (roundtrip at about a second), and the 60 commits took two hours. svn does not seem to like slow links; a simple tag operation (svn cp trunk tag/some) on a small thingy took me more than a minute over GPRS once.

Here's the complete commands:

mkdir myproj-cvs
cd myproj-cvs
git-cvsimport -p x -v -k -a -d :pserver:krey@localhost:/opt/cvs mystuff/myproj
git-format-patch -o out --root `cat .git/refs/heads/master`
mkdir mod
cd out
for i in *.patch; do sed -e s:company_name:companyname:g $i >../mod/$i; done
cd ../..       
svn mkdir http://localhost:4080/repos/mystuff/myproj -m 'base dir'
svn mkdir http://localhost:4080/repos/mystuff/myproj/trunk -m 'trunk dir'
git-svn clone -s http://localhost:4080/repos/mystuff/myproj
cd myproj
git am ~/myproj-cvs/mod/*
git svn dcommit

Inband signalling is evil

2008-09-22T18:09:00.003+02:00

In telephone inband signalling is, or rather, was to control circuit setup and teardown by tone signals within the speech band that is also transmitted from speaker to listener. Had the unfortunate effect that you use a specific whistle on the phone and lose the connection. Soon got exploited in more creative ways.

Anyway, in C++'s std::string often the same thing happens. When in C, you have a char*, and you can null it to mark the no value case. Not so in C++. You need a special value, and usually this is the empty string "". Fine and dandy as long as you can be sure that that value will never actually occur. Once this happens all the code will already be riddled with if (val != "") and you never find them all. Bad luck, just like with Y2K.

Neither do you have Maybe, the Haskell way of avoiding null pointer exceptions completely.

Dr. No vs. anonymous functions

2008-02-15T10:47:00.003+01:00

You may remember the first James Bond movie where the hero gets into contaminated areas and thus gets the decontamination treat. Now, this doesn't just happen but is shown in quite some details. As if to show off 'hey, look, we know all this stuff' instead of simply assuming it.

Now there is a similar thing with the syntax of anonymous functions. In some languages it is like function (x) return 2 * x; end while others say the same as \x{2 * x}. The shorter syntax stand less in the way and makes it possible to write, say, list.filter (\x{x.city == quest}).map (\x{x.zipcode}) in one line, while the former notation will cause massive keyword clutter hiding the actual operations.

There is a case in that the longer syntax may have been judiciously chosen to make clear that the use of anonymous functions (plus closures) is not encouraged in this language. I don't assume that it is a case of show-off, as in Cobol which needs to make clear that it can DISPLAY or COMPUTE something, as if this wasn't to be expected of a computer language. But the question remains, whether, in any specific language the form of anonymous functions wasn't just chosen because of its similarity to named functions. In named functions this is no big deal as their opening has a line for itself anyway. Anonymous functions need to be unobtrusive to be really useful, while the more verbose forms make many uses unwieldy. The question is whether the language designers really wanted to discourage that use or whether they didn't think quite that far in that direction, not having actively used functional style themselves.

The other ternary operator

2008-02-04T22:57:00.000+01:00

There is an interesting stain on OO languages like C++ and Java, namely with member function invocation. Normally, complex expression can be taken apart, and the intermediate value stored in a variable. int a = 2 * (3 +1); can equally be written int helper = 3 +1; int a = 2 * helper;. However, this breaks with obj.meth (). When you try instead to say helper = obj.meth; helper ();, you will run into a problem: What is the type of helper? It should be a function pointer/reference. Java does not even have functions; in C++ you are equally at a loss for a proper type. (I'm not going into that actually the parameter types aka signature form a part of the method name.)

Indeed the compilers simply go to treat the combination a.b(c) as a single operator. In Java the method and member name spaces are actually separate; a method can have the same name as a member. (Does that make it a Java-2?)

The only language that makes this explicit is Lua. Member invocation is obj:meth(arg), which is the same as obj.meth(obj,arg) (except that obj is only evaluated once).

I don't know of any language that does it properly. Even though a member function exists as code only once, logically there is a distinct function for each object. The member selection needs to bind the function code address and the object pointer together, into something that is quite similar to a closure.

Custom syntax: Macros and closures

2008-02-04T21:58:00.001+01:00

I'm lots interested in getting custom syntax for this and that where I get annoyed by C++ or especially Java simply not letting me doing any. For example, in Java you need to write all these EJB-style accessors yourself. In ruby you just say attr_reader :attr1, :attr2, and when reading other people's code you can be sure that there isn't a typo that let getAttr1 return attr2 by accident.

Just as I was embracing the idea that macros are absolutely necessary for much stuff,
ruby came along and brought block arguments. And tada, you can do map and everything without macros just as regular function. Some guys even ported the idea to java. So I became unsure whether macros are really necessary. Ok, basically I still believe that because (programmatic) macros can do anything; I think they are the best way to integrate custom code generators.

And then came along Raganwald and wanted something like ph = find_whatever()&&.author&&.phone: Call find_whatever, and if it returns something, get the author, and if there is one, get his phone number. Otherwise the whole expression is nil.

Now it is amazing that you can get ruby to do something like that at all, even though it cheats on two points: It does not use the intended syntax (obj&&.name) but obj.andand.name (note the conspicious second dot), and it does not actually take the shortcut but instead goes through the rest with a dummy object.

But he does not just to avoid the helper variables, but also to get a left-to-right reading. You just can't do that in lisp, for instance. (Ok, it's going to be the inverse.)

But why this hit me: While I have somes idea how I would bring macros into language with syntax, I have no idea yet how I would make &&. definable. The problems here:

When you get to the &&. you are already in the middle of the expression the macro would need to replace, so there would be a need to allow operators be a kind of macro

Member selection is not a binary, but a postfix operator! While it looks like one, the right hand side is not an expression, but a compile-time constant. (In a sense there are many different operators, one for each possible member name.)

I guess I need to think about that some more...how to enable user-defined operators not only as functions, but also as macros.

Burned by override: Limits of refactoring by compiler

2008-02-02T21:53:00.001+01:00

In OO programming (here: java) there is the difference between redefining a method in a subclass, and adding a new method in a subclass. It also is quite easy to accidentally add one instead of override one by having a subtly different method signature.

Then there is 'refactoring by compiler' as I call it. Just do the intended interface change at one place and be guided by the compiler errors to all the places that needs adatptions to that. In sufficiently static languages that does actually work (and it is a bit hard to get a program that compiles but does not work).

Anyway in this case I changed, compiled, fixed, and the program did not work in very strange ways. Well, it was the case from above: I started my refactoring by changing the signature of a overridden method, thereby making it into a new one that never got invoked. The macro did no longer think it was one, it went downhill.

Good thing scala requires to declare whether a method is an override. Java does too, but my personal java-fu is from times too old to actively use @override (or to sprinkle code with finals, for that matter).

Back to coding...

Giles Bowkett: IRB: What Was That Method Again?

2007-11-03T22:01:00.000+01:00

Giles Bowkett: IRB: What Was That Method Again?

Well, #methods is a good start. But it doesn't capture the method_missings. This shows just the power as well as the shortcoming of ruby and rails: You can't look everything up, you need to know some parts.

On another note, I'm just working on a small code generator that turns a protocol spec (of a proprietary thing) into java classes that offers the protocol commands as connection object methods. It's actually the same thing: We decided for the code generator, because that way, the IDE can show what is there; were we using annotations, this wouldn't be possible and things more dynamic. This isn't the whole cause; performance (as of exceptions, not primarily reflection) also played a role.

Still, I'd rather like to do the generation within the java compiler, which would save us some subclassing.

The generator itself is not in java, but in haskell, because of parsec, and generally to do some actual hacking in haskell. I like it; it is very nice when you change some definition and the compiler tells you the places you need to adapt.

Quoting haskell

2007-10-18T22:14:00.000+02:00

Well, Haskell is getting really interesting. I always wanted a language that allows me to intersperse fragments of other syntaxes (especially for a message syntax; think s-expr), and on the haskell workshop was an interesting talk about how to introduce exactly that for expresssions and even for pattern matching.

Right now I struggling with the right way to approximate the same in java, and bump into acceptance problems from some sides, whether I go via annotations or some code generation.

It's not that haskell isn't still rather foreign on me. I think I have a hint of understanding monads, but this doesn't go anywhere near how to design them.

Blub at large

2007-06-24T21:07:00.000+02:00

There are actually blub programmers who are not too shy to boast their status. Not only that, they point to the very people explaining their misconception and don't get it. They also manage to create a website that tells you that you need to login only after you click the comment link, and provide no visible of means to create a login in the first place, which is the reason for me posting this here.

(Seems the site has been dead for a while.)

Trust and Guarantees

2007-06-24T20:59:00.000+02:00

Bertrand Meyer had me laughing for a change. Obviously trust doesn't mean the same in computer science as it does in normal life; he wants to build 'trusted components -- reusable software elements whose quality can be guaranteed'.

The department of defense actually uses the same misconception: They are initiating a program for verifying the functions of chips manufactured abroad, and call this program TRUST.

Now trust implies that we don't know that a system has a specific property; we assume it has, on more or less sound reasons. Guarantees come via control, at a price. It's the same as in project management; either you have people you can trust to deliver results, or you need processes to control the output (and hopefully only of those you can't trust).

Blogger oops

2007-06-24T11:02:00.000+02:00

Blogger does not give me the delete button, and it should also not have posted this twice.

Learning at the barber's

2007-06-24T10:53:00.000+02:00

I accompagnied my wife to the barber's recently. The cutting looks so easy, everybody should be able to that! (Just like software.) I kept looking and learned. Even though it looks like he is just cutting away, he doesn't. There are different movements for the different stages, and there are tricks to check if the (long) hair has the same length on both sides.

Learned: To make it look easy you need to know what you are doing and what you can do. The last part, your options, are the hard part. You can't just bash away (at the scissors or at the keyboard), you need the confidence that you can handle upcoming difficulties.

Shackles of Compatibility

2007-05-27T18:35:00.000+02:00

The Shackle of Compatibility means you can never recover from mistakes in language design.

Well, I don't think so. If you do an incompatible feature you could just make it possible to enable and disable it. For example the newest rage amongst Java programmers is to declare every variable and parameter final. This is intended to catch accidental (as opposed to purposeful) variable reuse, which is a good idea.

But it is questionable whether sprinkling the source with final isn't too high a price. Ok, you can't just change the language semantics to have final be the default. But, why not make it switchable? Add something like pragma "default_final_vars";, and enjoy the best of two worlds. Old code stays as it is, new code can be not cluttered.

Similarly, any language could make the subversion or misfeatures selectable.

Of course there is a drawback as you must be aware which dialect is in use. But you need to do that anyway since many of the mainstream languages look pretty similar to each other.

Speech recognition

2007-05-07T16:55:00.000+02:00

Giles Bowkett argues that every programmer should develop a personal style,
and Andrew Norris responds that this is not a good goal. He seems to think that this is akin to writing Fortran in any language.

I don't quite agree. Personal style should be orthogonal to the style imposed by the idioms of a language. It reminds me of speech recognition, which can have several outputs: First, the words spoken. Second, who's talking. Third, the mood of the speaker. I don't have experienced 'speaker recognition' (that isn't tied to idiom usage) in source code yet, however, and can't quite think how that would look like.

I do remember, however, reading a german book about electronics, whose tone was just slightly unusual. It turned out to be a translation from finnish.

Desperate...sort of

2007-04-25T22:46:00.000+02:00

I'm back from holidays, and I just scan the blogs and don't get started with work. A year or so ago I stopped reading usenet, and now it's blogs... Anyway, finally I did start work, and then almost got desperate over my brain not immediately wrapping itself around every line of code of that smallish programm I'm supposed to extend (in a way rather cross to its design).

At the same time I wonder why I read so much about Haskell et al where I should be wanting to do some work with it. I got it into my head to attend the haskell workshop end of september which is conveniently located quite close this year.

I even got the strange idea of giving a talk there, about solving a relatively specific communication problem in haskell. Problem: I haven't used haskell for more than ten minutes yet and thus doesn't have hardly any idea how to implement it at any level, the submission deadline is in mid of june, and I have sufficient urgent real work on my desk to fill most of may.

So if I'd be any serious about that I should be prototyping haskell like crazy instead of continuing to read blogs, shouldn't I? Seems not... At least I'm not alone with programmers block.

And they I am not sure about the motivation for the workshop. See exciting things, maybe get a gig, or both? I suspect mostly academia there, so not much of a commercial opportunity.

So many ideas, so little time, and drive.

Synchronizing

2007-04-25T21:10:00.000+02:00

It is a longstanding problem of how to really synchronize mail folder, schedules etc. between home and mobile computers. Again, I can't remember who observed that originally. But the problem is about to dissolve instead of being solved, simply by being always online and no longer needing multiple replicas of one's data.

Now, a similar thing has happened before: There has been tremendous effort to make precise clocks as these were needed to get exact coordinates in ship navigation. You were on sea for months and did not want more than a minute of error. Then radio came and soon allowed worldwide transmission, and the need for mobile precision clocks disappeared.

Spell checking

2007-04-24T19:28:00.000+02:00

A blog post completely irrelevant to this topic complained that some software's spell checker did not know firefox and some others. Reminds me primarily that I practically never used them. The wiggly red line just had too many false positives. I seem to have a rather good spell checker built in anyway, and I wondered what an external checker would do to the internal one.

I found a strange analogy. This year I had a hiking stick (of the old-fashioned kind), and in the mountains it still had snow. The stick proved very useful to keep the balance crossing small snow fields, or, more exactly: It made me feel saver, but it damaged my ability to cross them without its help.

Although this analogy is quite probably limping, I shall continue to stay away from spell checkers. And let haskell check my spells.

Drowning in the swamp of pixels

2007-03-15T21:55:00.000+01:00

Yesterday it hit me hard. I 'just' had to fix up the logging of an application (which enables it to reconstruct its exact state after a crash). Unfortunately, it wasn't my application in the first place; I just had to 'fix' it. And all kinds of annoyances came together...

It was a shared office, meaning occasional meetings and all kinds of small discussions happening right behind my back. Pretty much impossible to build up any concentration in these phases, not even for reading blogs!

Then I had my laptop at 1024x768 and the local PC with 1600x1200 on its glass monitor. Problem: Apart from some silence I needed the big screen estate for actually getting all relevant code in eyesight. But unfortunately looking at that monitor only made me wonder what was wrong with my eyes; I couldn't actually focus on reading. In the old office it was just working for me, in this new one it didn't any more.

So I was getting desperate: To be able to actually getting anything done I needed to head back home to the fullsize LCD screen. The alternative would be to start massive notetaking, which is a form of thrashing and would just kill productivity as well.

I never experienced either workspace disturbances nor bad equipment that hard before, and now I am considering to buy another LCD for that workplace. Probably some silencer, too?

Flying

2007-03-02T22:17:00.000+01:00

When aboard a plane, give me a seat and a window, and I'm happy. Give me a stewardess telling me to close the window so that the sun does not keep the guy on the other side watching the move, and I'm not happy.

Looking around in business class I notice that most people don't care at all that they're just experiencing mankind's dream come true. As Richard Bach said, this is aviation, not flying, and thus these people don't care just how they get to Cairo, only make it fast. And that happens to be by air. Since most of them aren't good at waiting for the arrival, there is also a lot of entertainment of diverse kind abord.

Who would book a flight just to be able to watch a movie? And yet that is exactly what happens: Sit down and turn on the flick, even one they wouldn't think of going to the movies for. Just because there isn't anything else to do. Remember: When you're bored, it's because you're boring.

The trick of flying isn't, by the way, to fling yourself on the ground and miss it, as Douglas Adams had it. Actually, that's orbiting. You need to stand high enough and throw yourself not directly to earth but a bit to the side, and voila, your in a rather elliptical orbit if you manage to actually miss the earth (and the atmospehere). Flying is simply to throw down enough air so that in exchange you can stay up.

How not to blog

2006-11-29T11:58:00.000+01:00

I just stumbled over a few micro-blog entries, like 'what is time'. Cool title, completely disappointing content. The guy is just asking his commenteers exactly this questions.

Let me put it this way:

If you have something to say: Blog. (And 'that URL is interesting' is not interesting by itself, thanks.)

If you have somthing to ask: Google! (Instead of additionally providing misleading google fodder.)

^X^S

2006-10-28T21:54:00.000+02:00

I wrote the beginning of the previous entry twice. The old mozilla managed to crash just while editing that post (and googling for the URLs). Ok, if you can't even remember what you wrote then it's not worth it anyway. But being a programmer I do not like repeating myself, or repeating the refinements of text; I even notice that it bugs me when I actually write code fragments on paper and can't just insert the argument I forget and have to rewrite the whole line.

So forgo the blogger edit window and use the friendly local emacs, where your idle process types ^X^S anyway after a few seconds' timeout.

Teardrops in the programmer's eye

2006-10-28T21:51:00.000+02:00

Tears are a sign of pain. Or of being deeply touched, the soul's indication of a profound truth (and pain of internal readjustment). I had three pages recently doing that to me. The first was a the end of LFM and LFSP, about people stopping coding because they were forced to use langages inadequate for their mind. I feel the beginning of that myself.

Second: Transformation by Steve Yegge. The guy can write, and does so a lot, but this piece has an absolutely beautiful final point. You may not agree with his conclusions but he is interesting reading. And read the book first.

Third: This piece. My detector slightly beeped all the way through--there must be something to it.

Rigging your Poll, #2781

2006-10-18T21:22:00.000+02:00

A majority of germans would accept more video surveillance in railway stations (german) to fight terrorism. Says the headline. Now, the actual poll asked whether said surveillance would increase security. Note the difference?

I am actually quite convinced that monitoring every fart of the population would will provide less opportunity to do stupid things. But this is a far cry from whether I would accept cuts to my personal freedom, which I don't. Just look at the numbers. Getting innocently killed in a car accident is still must more probably than getting killed in a terror strike, even with 9/11. But for some reason nobody wants to cut driver's freedom to make traffic safer. Strange, isn't it?

Also, I fear to some extent that massive surveillance will not actually prevent terrorist acts. Even the israeli don't succeed in that. What will be easier is to identify the terrorists after the fact which is pretty pointless with the current crop of suicide terrorists.

At least I'm not the first one to notice, so I don't need create an account with them to be able to file a comment.

You shouldn't be reading this

2006-10-13T23:16:00.000+02:00

Well, this blog is just of the kind of dual-use that Steve Yegge had in mind: Making up one's mind and training writing. So as much as I would like readership I can hardly expect them. I like reading Steve's blog; usually they are long essay-style texts carefully crafted, and also happening to be in my line of thought yet inspiring. And he gets to points from seemingly unrelated starting points which I think is much better than just stating the point. At least when it's written in an entertaining way, which he usually does. No idea how much work he puts into that stuff.

Most other blogs feature rather shorter texts that just pound out a point or two (or even just an annoyance without any actual positive ending); I seem not to like those as much. Good thing about them: technical solutions presented there are preserved for googling, which sometimes made me want to write up stuff that I found out myself and isn't quite in any howto, for example.

For me, blogging work seems to happen primarily at late weekend evenings (when the next day need not start early). Don't be fooled by the posting times, by the way; I'm probably not sitting in the same time zone as you are.

Certificates

2006-10-13T23:06:00.000+02:00

I just had to sign a few of these. You know, the kind saying 'John Doe has successfully slept through "Incompatibility obfuscation complication library"'.

I could hardly keep a straight face through this. For, first thing, I just basically gave a talk along some slides, and, unaware of the certificates beforehand, I did not do any tests. (Wouldn't have if I knew, either.) Only indication of achieved understanding were the questions I got underway, which for some people were interesting, for other largely nonexistent, which again has many possible reasons.

And what does 'successful' mean? Has been there and awake the whole time? May be able to clone the demo examples? Has grokked the purpose and intent of that library? They first two are somewhat easy to verify; the third would take me many hours. But only that kind of understanding is useful, only it does not come immediately when presented with the whole set of concepts; it usually takes time to 'get' it.

And foremost, what are these certs good for? Since they are generally only vaguely correlated with true understanding, they are only useful for people who have no idea on the matter themselves (think recruiters-using-grep); in an job interview you could more easily find out by asking a few questions if the candidate knows that library or would easily learn it.

But in this case the course topic was a proprietary library that no one outside the company uses, so the certs are as shiny as they are exactly useless.

Also you wouldn't really want to work for someone who asks for the cert instead of asking you whether and how much you know the topic. They show that they don't trust you, and instead they risk getting someone who learned just for the cert. If you ask for crooks you get them.

The only advantage gained by looking for certs is the same as by buying software: you have someone else to blame if it doesn't turn out so good. John Doe should have known the iocl; Microsoft should have delivered a working Word, it's not my fault. Unfortunately finding the guilty doesn't help you at all (unless you are SCO). It covers your seat, but even your seat would have been financed from the project that just went down the drain.

...forced to find out

2006-10-01T12:48:00.000+02:00

I just stumbled over why there is a word verification (aka captcha) which keeps robots from posting comments to these blogs even though only registered users can post at all: There were enough of those that post automated 'nice article, here's my blog'. State of the humanity: Disgusting?

And yesterday we were shopping for shoes. We came over a nice pair that was unfortunately only available in 36 (yeah, we have strange sizes over here) and not in her size. We then decided on another pair, and I wonder if we later will find out that we don't actually like them that much and instead have only transferred our liking of the first pair to everything in the shop. Well, at least they costed less than the lunch beforehand. (We also have strange habits regarding relative opulence of lunch and dinner.) State of the humanity: Easily fooled?

And would you believe that people willingly buy stuff that is more expensive and less functional just because it is a bit better looking? Ok, it happens with shoes all the time, but I'm talking of glasses now. Last round I selected an open frame, which means no glass lenses because of danger of damaging. So, plastic was required, and the clerk quickly talked me into aspherical lenses which he said were thinner, due to being aspherical and also due to being available with a higher refractive index. A lot of time went by until they arrived (which is another story and not related to any complex production process). And then they turned out to be simply shitty. The regular glasses had a field where everything goes blurry, and the sun glasses had a bit of that nearly everywhere except when looking exactly straight through. Now, I intend to do viewing around by moving my eyes, not my head.

As it turns out, 'aspherical' is not the same as in photographical lenses. There the surfaces are not exactly spherical because their properties can be improved by slight, but properly computed deviations. In glasses, however, they simple start to flatten out the lenses from the (basically) correct spherical form so that they don't end up being so thick at the rim. This means simply that the lenses do not work properly in some areas! And you get sold that as an advantage: The lenses are indeed slightly slimmer, but more expensive and less functional. I wonder how anyone would want to live with that who has has proper lenses before. (Someone who gets her first glasses may however think that the imperfect correction is unavoidable, never having seen it better.)

Only good thing is that these days the shops offer full refund if there is anything you don't like about the glasses. I had other glasses made for the regular frame (at no extra cost), returned the other, ordered sunglasses in another shop (also spherical), neglected to specify exact kind of glass and thus spent an extra minor fortune on those, got two glasses that work everywhere I can look through them, and learned a lot more about spherical, cylindrical and prismatical refraction, eye distance yaddayadda that I wanted to know. But you need to know to get glasses you want to wear.

Funnily I hear a lot of people complain about colored borders (aka chromatic aberration) but I hardly ever notice those, and I don't mind them.