My first Mechanical Trading System results

As you may already know, my "new hobby" are Mechanical Trading Systems. After first quarter of executing transactions generated by my first Mechanical Trading System I have gained about 0% (literally almost ZERO). It may be surprising for somebody but it looks not so bad (but could be better).

Here is why:

• There were transactions costs, low enough for about 40 transactions, but still were.
• There were execution price slippages - sometimes it's hard to execute order at given price
• Some initial errors in systems gave me some loses too (about 10% of initial capital).
• I've made cross-validation procedure, but testing data set was too small to get good statistical meaning.
• Tested gain values were low (but still positive) and comparable to values without losses caused by "external" errors.

Statistics showed that the system was performing better in training period, and worse in testing and then in real account execution period. Underlying model was extremely simple and had above 100 transactions per year to avoid perfect fitting to one or few best price movements. But that is not excluding potential over-fitting on some kind of higher level longer horizon component of price movements, like up/down trend during sub-periods.

Results are definitely worse than gains from risk free return at a bank account. But it was relatively low cost exercise and I have learnt something new about practical details of investing, trading systems and statistical validation. Maybe next approach would be better.

So, I'm doing research on new simple model with higher gains/lower drown downs in test set. This time I have to make better validation tests to be more sure that system has generalization capability, and in future it could keep assumed risk/gain. That means more testing data and implementing better search procedure.

Calendar reminds me of plans for the future

End of year is good moment to rethink what happened through past months, and redefine some goals for the future.

Christmas and weekend days off, make good opportunity to take some time to think about it (besides family time and partying with friends).

I'm finishing older projects so there will be time share for new ideas. I'm doing some research and trials in Trading Systems, and I'm going to continue that. There is some place for personal development too.

I will be looking also for ideas for something more sustainable like startup project that could survive. It should be interesting. And fun.

Reviewing printed vs on screen documents

I have observed interesting phenomena: reviewing printed document reveals more mistakes in text than looking on the same document on computer screen. And I think I'm not alone - I've seen other people had same problem too.

It has to be something with focus on a task of document reviewing and ability to concentration in given context. Computer screen vs paper document. What is this all about? Is paper document better to read? I think it's related with casual context related with each of the media.

My computer (screen) context is something like:

• work (reading, writing, drawing, programming),
• news,
• learning,
• entertainment (movies, games),
• sharing some functions with TV,
• promotes multitasking,
• contains more chaotic and less quality content that needs more filtering and preprocessing

Paper context:

• books (facts and fiction),
• documents (more important and less like just printed electronic versions for convenience),
• better quality content that needs focus.

Time spent at each media is about 95% for computer screen and 5% for paper, so rare paper usage can make it potentially more interesting for brain too.

While using computer I'm doing many things while multitasking, and filtering low quality information that is reducing deep focus. Paper media in comparison switches my consciousness into context that makes me more deeply focused, less distracted (no multitasking), and and that makes me finding fine text errors easy. I don't even need to think about it, it's how my brain programmed itself in choosing proper context. It's not strange that computers (and Internet) turn me into "information trash" mode when I can't see details.

I'm not mentioning eye-strain effect, because of good modern LCD screens and readers using electronic paper so impact is small and difference between media in that area is vanishing.

Idea, how to stay focused on text without paper version of document, is to read text in simple non distracting computer environment like full screen mode, no other windows, bells and whistles (easy), and treating it like high quality content with meaning (that's harder). It may need some time to rewire brain, but probably that's the way to go.

Genetic Programming for finding trading system algorithm

I've been thinking about using Genetic Programming (GP) approach to find new trading system algorithm. There is no simple relation between such algorithm and outcome of the system (profit/loss benchmarks on historical data). To evaluate usefulness of algorithm, it have to be tested on real data. So its good target for that kind of automation.

I have been using Genetic Algorithms (GA) and even built some kind of GA framework at the university. Now I'm browsing through resources to refresh basics and find out what is new in the subject. I'm reviewing also GP frameworks to use (choosing possibly the simplest one). Results could be interesting - but it needs some integration work to take off.

Chrome OS - the dawn of net(only)books

Google is making good strategic move to get control and share of netbooks, and maybe nettops market. The incoming era of ubiquitous mobile internet connection for everyone, makes concept of thin clients and network systems plausible. The idea probably will be mainly accepted by mainstream population (look at software usage patterns and emerging fast booting systems from BIOS on desktops for basic web/email operations). If the big G stays behind it so we can expect hardware producers answer. Looking at the Android history it can be massive adoption of "new" OS concept.

There were similiar succesful concepts like Eye OS, but I tented to use separate specific web services best suited to one task at once like email, documents editor and sharing etc. Google already has those applications, so its also reason why it could be available as Chrome OS asset.

Some experts are calling that future is only in weba pplications. But some more advanced or geeky users it could be wrong way. It's ok to have most of tools as online service, but there is still place for lower level native (and offline) platform applications. Either Chrome OS "native applications" will be sufficient then, or Chrome OS will be installed as second system on more "fat" devices.

One thing is for sure - current web applications technology is still evolving, so this is only beginning.

Is OpenOffice good tool for advanced applications?

I had to make some "serious" and personalized reports in "office" format. One time task, no need for fancy reporting software.I had to acquire one of the MS Office licensed copies or just use newest OpenOffice 3.1 which I use everyday. I'm a fan of Open Office as a tool for simple documents and spreadsheets - there is no real advantage in choosing other solutions. But new task was more demanding: preprocessing table of data, making nice template and then generating personalized documents.

Data part preparig was rather tedious - fixing some errors, cleaning and adding missing values. The OO database fits well in the OO environment but it lacks more advanced data operations, like joining two or more tables, grouping etc. So I decided to switch to external database (sqlite in that case) and connect it to OpenOffice through ODBC Driver.

I have tried to make nice report with graphs etc., but I couldn't find a way to make it work with mail merge.
I used some scripting to manually copy template and insert some data, but unfortunately I couldn't set up some details, like print ranges and merging documents into one.

I dropped graphs and just placed more data in simple tables. That kind of document was easy to personalize using "Mail Merge" feature. I was afraid it wouldn't work well, remembering bugs in OO 2.x and hacking to get work around. No more problems this time. Good work OO folks!

It took me more time than I planned due to finding ways to do the work, but I'm still the OpenOffice (and family) proponent :). There are features I didn't even mentioned like possibilities of external scripting in server mode, making it even more universal tool (for hardcore coders).

Fantasy Kommander - tactical turn based Flash game

It's a games genre I like: tactical decisions with a bit of strategical planning. Isometric look at the battlefield reminds me graphics from XCOM U.F.O series, but placed in medieval times with typical fantasy races like goblins, dwarfs and elves. You can buy new units every unit has a lot of stats and characteristics that could be upgraded with game money. Turn based engine is simple- your move then opponents move. Units movements are animated and the whole game looks nice. Units could take counterattack also during defence phase. Battles look more simple that description. The best tactic is old "circle around and attack from behind". Many levels are very time (turns limit) constrained that makes playing even more fun.

I have found Fantasy Kommander title at the Kongregate, but it could be hosted at other sites too.

Build your army and go for the victory!

Post announcements on Twitter

A little bow towards Twitter fans. I've created Twitter account for IT Pro Life post announcements.

Still testing! Keep in touch.

After 100 posts

I haven't planned to get that that count, but it happened. When I'm checking readership statistics it looks like not a big achievement, but there is always somebody who can find something interesting for itself. The best thing I get through blogging, were contacts with interesting people sharing interest on common subjects.

I'm not actively generating inter-links, track-backs and using other web marketing tools. Most of the readers get here through search engines. After 2 years I'm still treating it as low time consuming experiment. It's not so bad, but always could be a lot of better.

Initially I've made routine to post once a week, giving yourself free days on vacations or during "busy" time. Now I'm bending that rule posting at less frequent rate. I'm looking for more interesting subjects, or I'm just more lazy. It looks like there is a trend to write less but more frequently (Twitter frenzy). Maybe I will have enough willpower and time to develop my blog beyond that.

Running an automated trading system

Some time ago I tried to get into automated trading systems. I have choose custom software trading platform that I'm co-developing with my friends that like system investing too (and helped me to quickly get into subject - so thanks a lot). It's rather my hobby than real source of income, but who knows.

My target was easy system, trading on index futures. It should be suitable for individual investor, that spends no more time than couple minutes a day on-line "looking at" the market.
After lots of tests (I really mean it) and choosing one candidate model I started using it at real account in "production" mode. Decisions are made by system, then manually validated and executed. The software was tested a lot, so no surprises from that side. Besides of that I still need to manually validate correctness of input data.

It's hard to get good free data source. I have heard that many commercial services that provide stock market data, have glitches in their data too. It's hard to find something usable for low scale individual investing - that kind of services are more oriented towards bigger businesses or "investing houses".

I'm still learning so I'm going to make next trading system after I catch some problems with the first one. My next target is more sophisticated system including machine learning methods, and maybe better procedures for choosing good candidate systems during backtesting.

How to eat spaghetti (code)

I'm thinking about dinner so the subject has a culinary style.

I have spent some time fighting with spaghetti code that grown from adding more and more conditional strands to initial prototype of application framework.

Before major version release of middle size production system I had to add a new feature to the biggest important process. Not so big change - it looked simple. It just involved doubling of some model objects and making two separate subprocesses. After alternating main flow, many side effects occurred, caused by intertwined logic and persistence. I had to do some clean up, cut off tangled strands, standardize conditional flows and reduce code. The results still weren't valid - at least I have found "single point of failure". Because of time constraints I had to fix it with block of ugly code. The project needs change in the process structure to adapt new features in more seamless way.

When you meet that kind of "spaghetti code" there are two known ways of dealing with it:
- pull one strand at a time - clear one flow path - repeat until structure is simple enough
- eat it all at once - analyze requirements and used implementation and rewrite it from start

Reduce structure, simplify, refactor and do not let the code grow into spaghetti again. It will ruin maintainability and extendability of software project in future.

Scatter chart library - moving to Flot

After playing with Open Flash Chart 2 trying to make interactive scatter plot, I've found severe glitches that popped up during prototyping. Library worked quite well for standard line and bar charts. But for my specific scatter chart I realized that:

• I have to use hacks to provide 0,0 origin based XY axes (OK I could just use standard left/bottom axes)
• there are problems with larger scale values (I didn't checked exactly but values in range to 10000 rendered OK, somewhere above it is some kind of limitation)
• problem with grid rendering for values in less than zero range
• OFC can't parse scientific notation values (data number formatting is a must)
• some minor problems that could be solved by chart tuning and hacking ranges etc.
• after playing with OFC stable version I've checked last trunk version that worked much better

I summed all problems and thought about potential extendability problems and tried to move to library based on HTML Canvas object. One of the simplest and most mature projects is Flot. It has all required features, including interaction. The only problem I found was rendering under IE using Excanvas emulation object. Standard stable Flot version doesn't work well under IE8. Charts were drawn although without data points. After checking project issues list, I've found that some problems where resolved by using newer JQuery and Excanvas library. I've tried to run Flot from trunk version, and my IE8 problems were solved. As a extra newer Flot version contains more interesting plugins like navigation. BTW maybe it's time to publish next Flot stable version in downloads section. The only disadvantage of that solution is forcing IE users to accept running Excanvas security policy.

Long break and back to work

It was long time ago from last post. I had real "without thinking about work" vacation. Mostly cycling and resting with family in small house near lake shore. I've been using Internet mostly for communication and weather forecasts. After back to work I had to catch up to meet approaching deadlines. So sorry about not giving any signs of life for a long time.

Choosing scatter chart library for web application

So I definitely know that I have to make web based UI for medical diagnostic application. It's not so complicated as I initially though. It is a kind of scatter chart that has to display delivered XY data and use as reference some kind of meta data like description, database id etc. Basic functionality of that module is adding and editing points. It should be somehow interactive - like has clickable points that can fire various events. Graph zooming is also needed.

There are articles with list of interesting solutions: 13 useful graphing solutions, 10 Free Chart Scripts. I reviewed few charting libraries that can be divided into three categories:

• HTML canvas charts
• Java applets
• Flash based charts

There where also other interesting ideas like SVG based interactive chart, or image chart with imagemap and added DHTML tricks, but still without enough support and needed writing more code.

Needed charting solutions should be lightweight so I thought about html canvas drawing, but unfortunately there are serious IE issues - It does not implement such feature, so excanvas ActiveX is used not always successfully (new IE changes are breaking excanvas functionality).

Another good choice from point of customization and extending functionality is Java applet based on JFreeChart, or lighter jchart2d library. Using Java applet was tempting. I made some serious applet work some time ago so I had to make some user experience tests of deploying UI as applet. The large group of users has no technical computer background, and can be confused about accepting prompts for Java installation (Java is not preinstalled with system in here), waiting for runtime download and finally for applet initializing. That may cause serious problems with acceptance tests.

So I tried Open Flash Chart 2- the nice flash based chart library. It has fixed but all basic features I need. Flash player is king of the browsers (large deployments ratio) today. That and smaller than Java's runtime should cause less problems with runtime installation. It has wrapper libraries in most popular programming languages including Java. I have choose third party jofc2 library for quite easy integration with code sample from Open Flash Chart and Wicket example.

Open Flash Chart has enough javascript interface, to dynamically change data, graph style, add tooltips and script onclick events on data points. It has no direct zooming support but can be implemented as changing chart properties by adding external scripted controls. Redrawing after data delivery is fast also.

Near future brings even more choices like Silverlight, improved Applet/JavaFX runtime or adoption of canvas like solutions.

Further reading:
Scatter chart library - moving to Flot
Looking for simple python chart library

Rich GUI dilemma - web or desktop application?

I'm planning additional visualization tools for diagnostic system. The have "additional" role - because will be scarcely used only by expert users. It's a kind of fancy interactive scatter plot with plenty of displaying options including simple data transformations.

Primarily I thought about it as a desktop application. But I when I get initial requirements it seemed that there is no much functionalities and it could be written as module of already existing web application for data gathering.
I have to gather more details , because now I have to guess future boundaries of that part of the system.

The basic pros and cons array:

	Pros	Cons
Web application	- its web application so you need only web browser (maybe Java or Flash runtime automatically downloaded by browser)- easy deploying - better (seamless) integration with the rest of the (web) system	- possibly low client performance depending on used technology
Desktop application	- established for rich GUI - more computational power to use on client side (really?) - targeted for more functionality "richness"	- target client system dependence (but I know that 99% of users uses Windows platform) - loose integration with the rest of web based system (deployment architecture allows it)

Reviewed technologies and solutions:

• Java desktop app
• Java applet
• GWT application
• YUI components
• flash components

Final choice depends on the range of features, and final users preferences. Web version seems more performance constrained - but there are many technological solutions that make that choice as good as its desktop counterpart. At that point there are very little differences between web or desktop deployment outcomes. Nowadays buzz is "web is the future". I have to make my choice based on economic assumptions.

The best programming language 2 - Simple case tale

The University needed temporary solution for generating list of quiz questions, chosen randomly from data prepared in structured text file. The scope of application was nothing comparing to "Enterprise Systems", so I treated it as a kind of entertaining enterprise.

The core characteristics were:

• simple logic
• UI forms for getting parameters from user
• printable or saveable formatted questions list
• low burden installation and use (get it on pendrive or download ready to use).
• main constraints: short time and possibly low cost.

I started looking for similar open source products, but not found needed functionality. So when I checked myself for Not Invented Here syndrome, I started thinking about custom solution.

There were many ways of implementation. I thought about preparing windows exe file with included required dependencies (no installing additional Java, .Net, Python or other runtime).
I started playing in "prototyping with Python"- it was fun and things were going smoothly. But I stopped before making UI. Plain old console was "too lame" for final users, so I planned GUI with TkInter included by default in python installation to avoid porting to windows executable problems.

I get core set of functionalities, but user interface still was barely specified. Then I get many additional requests about "last minute" changes in UI forms (and probably not the last). The second "economic" option was to implement the whole application in Java Script and HTML running under web browser. Every computer has web browser suitable to run that kind of application so there are almost no installation and portability issues. Implementation time should be short too. To speed up things I have changed data format from flat file of records into JSON structure. Porting logic was fast too. The most time took me making forms working.

First version was not so fancy and needed some GUI tweaking like formatting, changing labels and headers. In a case of any further UI changes other people knowing HTML could make fixes instantly. The extra feature was possibility of "deploying" the little application on a web server as HTML page. Saving result to file wasn't working directly. Making it really automatic needed interacting with browser security policies for access to local file system. We decided that it will be still usable if user use copy-paste technique, and place formatted result in external text editor. The default usage was printing questions on paper so nobody saw problem there.

Everything is working as designed, and shows that web browser should be treaded as good platform for small applications. Sometimes little prototyping gives better insight on a whole system.

Who are my readers?

I want to get in touch with my loyal readers, so please share with me some information about your interests and suggestions about this blog.

I hope it will help me to improve the shape of this blog or make connections with interesting people who has wonderful ideas and make cool projects.

Post answers in comments or use my blog email: itprolife@gmail.com if you don't want make it public.

Here is short survey template

Nick/name:

Country:

Your website/blog/other web realm:

Favourite websites:

Interests:

What do you like/hate the most in my blog?

Other suggestions and comments:

Contact (optional)* :


* all fields are optional , so feel free to write anything you want

Pragmatic printable document format

What open file format comes to your mind when you have to produce mass of personalized electronic documents in enterprise system?

I have reviewing pragmatic solutions for given requirements:

• ready for batch print out and producing quite good prints on customer's printers
• keeping documents in electronic version for evidence (content cannot be changed)- the bussiness process is paper based
• have customizable templates (text, formatting, tables)
• total cost of process should be optimized (including printing by customer issues)

PDF seems good choice, it is well distributed format and produces nice print outs. I've tried it and it worked quite well. The cons of that solution popped out during batch processing and were: slow process of generating final PDF and bigger files included containing additional font data (national character issues - no problems with standard fonts).

Another approach was using XHTML as final format with ability to print from web browser. That needs more attention on formatting and automatic flow, but it makes good prints also.

Users can print documents using both formats without problems. XHTML was faster to produce and worked with standard system fonts, so files were in most cases smaller than their PDF counterparts.

That was scenario for paper dominated document flow process. Electronic document processing requirements need also easy and automatic access to important data in document content. The best standard for that seems XML containing structured data. That plus XSLT transformation for description of form view, may be used to produce printable XHTML or PDF (with help of FOP (Formatting Objects Processor)).

New document formats like ODF are based on similar ideas so future lays somewhere here, just new solutions need more adoption.

Removing page from Google index and cache

Everybody wants to get search engines attention for their public content. But there are situations when something improper leaks to public by mistake, and then we are trying to reduce harm caused by an incident.

That happened to one of our clients so I get a request to remove particular page from Google index and cache. Of course it wouldn't help much for content that was hosted for about 1.5 year - if people found it interesting, there is practically no possibility to remove that content from the Internet. But most people are seeing only Google results so decision was to target Google.

It seemed simple and obvious. The possible methods are:

• filtering page by robots.txt
• meta tags for bots in page html
• 404 http answer

Google support pages has detailed procedure about removing individual pages from index and from cache.
There are obvious assumptions that could be easily forgotten. Your site needs to have valid robots.txt file or be hosted without robots.txt. In case when the Google crawler engine assume that something is wring with your robots.txt it even don't move any further to update your site index or cache. After some tweaking it seemed that case is ended. I registered urgent request removal on webmaster tools. And waited for crawlers.

Everything should work as planned? Wrong!

After few days I get my "urgent URL removal request" denied. Reason?

"The content you submitted for cache removal appears on the live third-party page. ....
As you may know, information in our search results is actually located on third-party, publicly available webpages. Even if we removed this page from our index, the content in question would still be available on the web"

No clues about what that third-party page is. Probably crawler engine can't resolve issues caused by multihosting on my server plus couple of DNS names for requested site. I'm still trying to get more information about problem.

UPDATE

Adding second domain to Google webmaster tools helped. That time I requested page removal from cache. Indexed URL is actual also.

Time is the simplest thing - what is the cost of leisure time

First and basic rule: our personal time is scarce resource.

I realized that I'm playing in one of online games way too long- it took me about half an hour per day. Despite of fact that at the beginning it was really fun and now its sort of social experiment with gaming background, it's just leisure time. Everybody needs some relax but I felt guilt for not using my time in more productive way.

I've quickly calculated my often used average wage per working hour and applied that to time saved by dropping some of my leisure time. That's the coarse result that says something about earning more money instead of spending time in other unproductive way. But reality is more complicated than that simple model. There are other factors that should be also considered:

• conversion of free time into work time is a must (what if freed time will be used in another way?)
• extra costs of additional work
• demand for our work services (supply and demand effect) could change our price
• costs of shortened leisure time (too much work without rest could cause bad results, like burnout)
• effect on life goals

As a result, limited time forced me into thinking about balancing life activities. Sometimes money is not enough to sacrifice more personal time for work.

What are your way of dealing with that kind of situations? What is your time worth? Any tips for balancing work, leisure, family time?

Spring break

In April I broke my weekly posting schedule. Don't panic - it's only a break! Taking few days off at Easter time, spring cleanup and regeneration, and then back to heavy work when one of the clients backfires with new features request - I lost attention for some of my routine task. At least we have nice spring, that makes me feel better (my bike is ready - now it's time for me).

I've checked readership statistics and unsurprisingly it dropped by about 30%. It only confirms that people (or bots) are looking for fresh content.

I want to thank to all my readers, the very specific ones that not get bored easily reading this stuff. It is the last thing that keeps me writing.

More feedback is appreciated - so leave your comments.

How to get (rid of) Anemic Domain Antipattern coding multitieried enterprisey application

Programmers want to make "good designed", "enterprise like" application. So keeping in mind boxy schema of multitiered application they start to organize code into separate layers like most common DAO,Service and Domain. Very often that way leads to Anemic Domain Antipattern. It's characterized by "dumb" C struct like domain objects without logic in methods.

The mechanism is simple: programmer wants to keep layers separate and dependent only on common interfaces. To reduce wiring and simplify initial development DAO object are connected to Service layer which uses also Domain objects. Service layer is filled by Script like methods, written in procedural style. That way of development is easy at the beginning. So main logic goes to Services "layer" and data operations go to DAO layer. After some time Services become really fat, but domain objects gain only more attributes and accessor methods. As long as application is a kind of a prototype, or just is small everything works quite good.

But to get more benefits of object oriented programming in future, we need net of more "intelligent" domain objects "talking" to each other, not just logic closed in services layer scripts. At this moment that could only be done by total refactoring, or just by designing from scratch using experience from previous implementation. Just keep in mind mantra encapsulation, reusability, simplicity.

The simplest cure to that antipattern is to design "good wiring" at the begging, to give domain object direct access to DAO layer, and other low level services needed by domain logic. Then, domain can evolve easy into more complex structures, needed by bigger "enterprise" applications.

Another point of view on Object Oriented Programming problems

It's not another article about OOP pros and cons.
I want to present you a non orthodox example of dealing with common programming problems without using standard OOP patterns.

The site is for some time on the web already, and presents controversial articles about OOP and SQL criticism.

I don't agree with author's overall point of view, but many ideas are very usable in specific simple applications development, and some of patterns presented there could be found in current RAD frameworks.

Just read and brainstorm.

Planning to build own cellphone based sleeptracker project

I'm using Sleeptracker, but I need something to get better data that it provides, and have more flexible work modes.
There are many interesting projects with custom devices based on micro-controllers. I used to solder small electronic devices in past, but now I'm maybe too lazy and just looking for some pragmatic device, that looks and works good. It's fun to make design of own device but there are many ready small computing platforms with suitable features.
I'm thinking about cellphone, or maybe smart-phone, with built in accelerometers and available free software development tools for developers. It needs to have accelerometers and API to get acceleration data from it.

That kind of device has nice features:

• it's programmable so you can put any features you think about
• compact size, good for wrist attachment (ok maybe little too big, but make some trade off)
• built in battery with quite long operating time and possibility of charging
• possibility to use other input devices to get data like camera to get light level, or microphone to catch noises
• builtin gprs or wifi to send data to server
• of course its still cellphone

The biggest problem may be lack of API to needed devices. New devices are more open than in those in past, but some vendors still don't expose access to internal services.

Switched from Blogspot to new custom domain

I have switched to new custom domain:

itprolife.worona.eu

I'm still testing it so DNS problem while getting to my site are probable.

Minor health problems - work is not everything

I caught a cold and get some stomatological problems. I was unable to make more complex tasks for a longer time, I just needed a rest. That focused me on myself and my health shape. Sitting in the front of computer doesn't make me fit and healthy.

My RescueTime report states efficiency score:

EFFICIENCY 1.58 for 28h 39m
AVG. USER EFFICIENCY 0.53 for 9h 27m


Maybe I have wrong RT settings, or my development and personal productivity categories kick in, as I spend a lot of time working. Maybe too much time.

How to learn simple programming in a fun way

I have looked for games that incorporate programming or learning programming and I found few interesting gems. One is SCRATCH visual programming environment for kids made by MIT with sets of basic flow control and sprite sound and animation commands. I have tried some tricks with LOGO like pen drawing features using drag and drop interface and it looks intuitive and appealing enough to catch curious kids attention.

On the other hand I found less appealing low tech c-jump board game with goal of stepping through C like syntax simple arithmetic program. It is simple but still shows the concept of execution of sequenced commands with flow control.

The direction is promising for Microsoft, as they are developing KODU project, aiming at children too. The concept looks like interface for BPEL kind of language and probably in future that kind of environments will be more popular. You can find there links to more similar projects.

Blending gaming with simple programming may be a catch for working adults too.

The art of exceptions catching

Exceptions in programming languages are very useful tool for propagation information about program errors and helping handling critical situations properly. But it is only tool and how it can be useful depends only on implementing programmer.

I can put it in few points what I have learned in theory and verified during practice:

• If you don't know what to do with exceptions don't catch'em - exceptions reveal errors so fix errors - don't hide them under carpet

• Catch exception if you can handle it, you know how to clean up used resources and how to inform developers/users about that event

• Don't use exceptions mechanism (hacky way) in the correct flow of the program

• In Java use unchecked exceptions (RuntimeExceptions descendants) instead of checked Exceptions - it saves "throws SomeException" litter in code forcefully reminding about catching them (it makes sense only in some kind of low level services with inevitable critical situations that can be handled by direct clients)

I think it's only basic set of rules.

Do you have another ideas for exception handling?

Thinking about summer HAR 2009

I've just read news about Hacking At Random 2009 event at 13 - 16 August 2009 in Holland. It's tempting proposition of spending some summer time in interesting and fun way. It's second after Chaos Communication Camp big European "camping and technology" event, full of interesting people. I have to rethink my summer time priorities and decide to go there or not.

Anybody is going there?

Improve PHP application performance using opcode cacher and optimizer

One of university underpowered server cought "hiccup problems" during students rush at local Learning Management System (leading PHP LMS platform) during the end of semester. Because there was no chance for quick hardware upgrade I've checked server tuning check list again. Many basic settings was optimized (caches, all additional modules turned off etc.), but there was still room for small improvements. The big problem was memory limit under high load.

I've installed XCache PHP opcode cacher (they say it's production ready), and set it to cache most of common used code in RAM. Results was better than I suspected, maybe because of that application is utilizing a lot of PHP code. Under heavy load memory usage dropped about 3 times, the same gain factor was for CPU usage. I've checked if everything works ok and left it working with XCache then.

The next bottlenecks were database and CPU usage for some operations. It's hard to get further performance improvements of that scale without tweaking application code.

Good SF book - Permutation City by Greg Egan

I have finished reading Greg Egan's "Permutation City" novel published in 1994. It's full of interesting ideas I liked or worked on in some way in past. You can find there consciousness simulation, AI living in VR, and other theories like cellular automatas and artificial evolution. The novel story is based on many fictional and unrealistic assumptions but the whole idea of creating computed simulation of an embedded then detached universe is very mind appealing.

The first kind of ideas in scientific world came from Konrad Zuse that proposed idea of universe computed on a computer grid. It's very strong paradigm shift of viewing our universe. Taking quantun effects and last popped popular theories like holographic principle, give me very strange feeling.

At the end read a xkcd comic with a twisted idea

Update:
Another research covering that area. Check the controversial but interesting theory by Stephen Wolfram.

Using simple drawing tools to unlock creativity

One day I have played with my kid's sketch pad toy. It's about 15 inches diagonal - almost like smaller PC monitor or laptop screen size. After drawing funny pictures I started to draw and solve some of my work problems. It was astonishing easy to work in that way, and soon I get simple solutions. I realized it is really good device to unlock creative possibilities.

I rethinked that again and spot some interesting factors that helped me to enter into that state:

• playing mood (it's almost as important as all other factors together)
• everything you draw or write is temporary, unless you try to save it in some way, otherwise you make one sweep and you have clear area ready for new ideas
• comfortable drawing with pen and handwriting unlocks physical obstacles slowing your work (like using keyboard)
• all factors shorten interaction on path brain - drawing surface
• the next similar tool is notepad with pen - loosing some of the play joyfulness

If we could transfer those features to laptop.. wait a minute there are laptops- tablet PCs! I've never used one but I'm really interested if is it as easy to work as simple sketch pad or notepad.

How to export data to one big XML file

My company had to add extra export format for data in a system I'm working on. I get specification and XML schema and started analyzing it for best fast solution. Data was ready to export so there left decision about choosing right way to make XML. Deadline was imminent, and everybody was really busy at work. Making export file - was good task for our newest programmer. He never parsed/created XML using SAX/DOM or other libraries, but was envy to use and learn one of the standard.

The main technological problem was the size of output XML. For biggest data case it would generate over 500MB XML with about 50000 nodes at highest level. It would be efficiently generated using only writing partially to stream technique like SAX. I have chose two options- use SAX or own similar simple API - realization time was crucial. Because XML structure was relatively easy - all tags without attributes, so we decided to use plain string buffer technique with addition of couple helper functions.

At the beginning I had objections about not using standard libraries that should help. But the implementation of XML format moved forwards, output file was easy to validate with XML schema. The most time was committed to handle non existing or incomplete data cases. The export function was ready before time. That is one of examples where using simplest tools, meets desired goal.

What are your choices in similar situation?

Successful simple web based MMORPG

Massively Multiplayer Online Role Playing Games are at the peak of popularity. People like to play with other live players and compete with them. Game producers are gaining more and more profits from that kind of games, giving better and better visual and audio experience in pseudo 3d worlds. Is there a place for a technically simpler web based games?

One of my friends told me to look at the one classic game of that type: Fallen Sword produced by Hunted Cow Studios Ltd.. Game started, as I know in 2006 and today has about 2 700 000 registered accounts. There is probably massive amount of inactive players, but the number is still impressive. As I checked during day there were always above 2000 players online. So people like not only WoW 3D style games.

The main game concepts remind some of popular MUD games, with added graphics, auction house and trade between players suport. You can buy special game money units, for real money using for example PayPal account. Those units called Fallen Sword Points maybe used to boost game character skills or other attributes. Although there is no way out for money as company states in terms of use, in game economics is quite strong. People are playing game for fun, but sometimes are trying to trade game money for little services, like giving temporary skills boosts called buffs, or "selling" fan-art avatars. Those gray areas are acceptable, in other cases the real winner is producer.

The game lacks some responsibility that can be gained through more AJAX like components, but overall speed is acceptable. I'm trying to figure out other interesting features - and ideas for game people may have. I like those kind of the non highest tech but very playable games. I had one web based game started and unfinished prototype in past, so I'm interested in subject from a business point of view too.

Dual boot system failure - Man and Machine tale

I have get a computer with relatively small HDD (only 80GB). It hosts Windows and Linux OS with GRUB boot manager. I needed more disk space so I decided to remove linux, and create new NTFS partition in that place. It looks simple. So I run the partition manager and checked what is what. First 100MB small partition looked like linux boot directory. Next was windows and then linux and swap partitions. Easy. Remove linux and windows partitions, format. All done. I played with Windows settings and decided to reboot.

Then I saw my mistake. I accidentally removed GRUB stage and config files with linux partition. The first one partition does not contain those needed files. I should check it more thoroughly. It sucks. I suck.

The first thought was, get the windows CD and rewrite master boot record with windows boot loader. But I have no cd/dvd drive in that machine. I don't have spare one in that location. But I have my laptop and USB pen drive. So maybe another option. I will install small linux distro as a power utility and reinstall GRUB. That's I like more.
The first thing I tried to do was creating bootable USB flash drive. I have used couple boot images, and Puppy Linux distro, different BIOS settings - my machine does not allow me to boot from USB. Not good.

So I decided to boot windows by manually configuring GRUB. After little research and experimenting I have booted into Windows. Now every time I want to boot that machine I have to:

• get into GRUB prompt by pressing 'e' key
• write GRUB commands

root (hd0,1)
makeactive
chainloader --force +1
boot

Nice "hacker" style setup. It was definitely not finished yet.

I next part I write how I made that system more usable.

Happy New Year 2009!

Begin of the year is the magic time of new commitments and promises.

I'm going to fulfill my professional goals and having more fun in life, doing that in a healthy way.

Greetings.