I’ve been doing a lot of code reorg, but things are starting to look good.  I’m also rethinking what’s really required by the program and am focusing on getting the necessities implemented cleanly first, before even thinking about some of the old “extras” like “mobile mode”.  I’ve also decided to, for now, drop development on the console interface and focus purely on the GTK one.

Anyway, that’s that.  To quote 3D Realms, J-Ben 2 will be done “When Its Done.”  No specific timeframe; I feel that’d be rude at this point in time with how little time I have.  But unlike Duke Nukem Forever, J-Ben will be released.

For family, I have some big personal news.  I prefer not to say it publicly right now, but if you ask Mom, she can tell you what it is.  Otherwise, ask me in an e-mail directly.  I’ll update this in a couple of weeks when it’s safe to reveal this.

For everyone else: well, I’ve now moved to Amagasaki, back in Hyogo prefecture.  I no longer live in a “gaijin house”, but rather have a proper two bedroom semi-new apartment.  Got a good price on it, and of course I found out a few reasons later; I’ve known that it was close to both a highway (overpass) and a railroad, but I did not know that it was close enough to the highway for “earthquakes” to happen when a semi truck goes across it at night.  Also, in winter it is rather cold; Yukari keeps complaining about it.  I’m sure she’d rather I found a different place, but considering how much money I’ve put down on this one, I don’t see myself moving so soon.  But those weak points aside, it’s a fairly comfortable place, very roomy for a Japanese apartment, in a halfway decent neighborhood.  I no longer live in a slum, yay!!

Then, on the development front, J-Ben development has of course been on and off.  Since I’m on vacation from work, I’m planning to get at least a working beta of J-Ben 2 out within the next week.  I have the basic individual pieces working fairly well; I just need to hook them together properly.  The beta will not have all the same features as J-Ben 1, but it should be a much quicker program.

I also noticed just today that J-Ben 1 does not seem to want to compile on Ubuntu Karmic.  Seems that the Boost libraries are not being linked in properly, perhaps due to being in a different directory or something.  More reason to eliminate the Boost dependency and get everything going with the Python version.

That’s all for this time – take care everyone, and have a happy new year!

Unfortunately, the default ExpatParser doesn’t have a clear way to disable this. But if you read the docs closely enough, you can find out about setting a “default handler” for it, which has the side effect of disabling internal expansion.

This isn’t a perfect fix, but here’s the class I used to get this done:

# Code is Copyright 2009 by Paul Goins
# Released into the public domain - but let me know if this is useful!
 
from xml.sax.expatreader import ExpatParser
 
class ExpatParserNoEntityExp(ExpatParser):
 
    """An overridden Expat parser class which disables entity expansion."""
 
    def reset(self):
        ExpatParser.reset(self)
        self._parser.DefaultHandler = self.dummy_handler
 
    def dummy_handler(self, *args, **kwargs):
        pass

So, use an instance of this class instead of one by xml.sax.make_parser (or xml.sax.parse, for that matter). Then, in your xml.sax.handler.ContentHandler, specify a handler for skippedEntry(self, name), since that’s where your internal entities are going to pop up. By the way, built-ins such as & and < still seem to be parsed; at least in the case of JMdict it seems to just affect the parsing of the inline DTD.

Hope this helps someone!

My life’s been pretty busy as of late, preparing for job hunting here in Japan. I will be continuing to live here for a while longer; I am unsure how many more years. So, I’m beginning to search for employment here in Japan as a software engineer. Also, I am actually being kept somewhat busy at my job as of late, which is without question a good thing, except that I do have less time to work on my projects.

Speaking of my projects, I do have an additional one now. I have become the official maintainer of GNU Typist, and have been working on it a little to prepare it for a new release. The next version should be available within a few days from http://www.gnu.org/software/gtypist. This is a lesser priority project for me, and mainly I’m going to try to rely on contributions from other programmers, however I’ll be handling support issues and will occasionally contribute code. Both the gtypist patches here on vultaire.net have been integrated into the soon-to-be-released version.

J-Ben has been sitting idle as of late, since I am trying to focus on job hunting as well as actually using J-Ben to study Japanese. However, I have a number of ideas for improvement:

* Add a new dictionary loader for JMDict-format dictionaries.
* Rewrite GUI so that the GUI will load instantly but be disabled while dictionaries load. Currently J-Ben just loads in the background, and on some systems this is very slow. I’d like quick feedback that the program is indeed loading. A splash screen would also work, but I don’t really care for splashes so much.
* Add lazy dictionary loading. Dictionaries are auto-loaded when they are needed, and not before. It will slow down the first search but should be an overall improvement, especially if the user only uses one of the dictionaries (without cross referencing between them).
* Add dictionary cache. A cache file would be loaded instead of the dictionary files, if present. It’s hard to get any faster at loading data into memory than this.
* Read data from dictionaries on disk as required. This may lower our memory footprint significantly at a (hopefully slight) cost in speed. Implementation may be a challenge since I’m supporting many dictionary formats, but if this code is based on the above mentioned cache instead, it should be a reasonable goal.
* Add multi-radical search to the Kanji search window.
* Tweak the resizing code for the Kanji search window.
* Replace GTKmm code with straight GTK+. Yes, this seems backwards, but it’ll save about 5.2M of hard drive space on installs. GTK+ itself is almost 8.75M, which is already a heavy penalty to pay. I’d rather not add more weight to J-Ben than necessary.

The above is a lot of work, and I’ve been trying hard to avoid getting sucked into it too soon. J-Ben as it stands is not so bad, but it does take too long to load and eats a bit of memory since all dictionaries are loaded into RAM for performance. I think this may make it less appealing than other programs to many users, and would like to, eventually, fix this.

Finally, I’ve been working on a small new project. It’s a program called NonoPaper, a “helper” program for doing nonogram puzzles, also known as gridlers or Japanese number crosswords. It’s written in C++ with GTK+ (not GTKmm), and it supports multiple color nonograms and partial-block nonograms (nonograms where the end of a segment may be a right triangle rather than a square). No solver is intended; this is meant just to be an application for doing nonograms yourself.

That’s a summary of what’s in the works. So, expect a new release of GNU Typist from www.gnu.org/software/gtypist in a few days, and maybe within a month you can expect a first beta of the nonogram program. J-Ben, barring any critical issues (or unexpected extra spare time), will be put on the back burner until probably late June or early July.

- Paul

However, I think there’s one more feature I want to add to that release, and that is making the English-Japanese search recognize English word boundaries. This will not affect Japanese-English searching; I may add restrictions in the future like 2-or-more kana on kana-only searches, but for the time being I don’t think it’s terribly necessary.

One last detail: I am pushing my code to a public git repository now. Repository details are listed at http://repo.or.cz/w/jben.git. Do I have a real serious reason why I chose git as opposed to CVS/SVN? Honestly, just that I would like to learn how to use git properly since it is what is used for Linux kernel development, and I’ve been curious how different it is from CVS/SVN.

That’s it for now. Oh, if anyone reading this checks out my Japan blog, I uploaded about 11 months worth of photos and videos to the media section last night.