Copied from http://oferia.pl/zlecenie/item499594-program-w-javie and translated:

 "I ask to write a program or game in Java. Program / ​​game must contain at least 15 classes and show all the stigmata of object-oriented programming..."

How to eat spaghetti (code)

I'm thinking about dinner so the subject has a culinary style.

I have spent some time fighting with spaghetti code that grown from adding more and more conditional strands to initial prototype of application framework.

Before major version release of middle size production system I had to add a new feature to the biggest important process. Not so big change - it looked simple. It just involved doubling of some model objects and making two separate subprocesses. After alternating main flow, many side effects occurred, caused by intertwined logic and persistence. I had to do some clean up, cut off tangled strands, standardize conditional flows and reduce code. The results still weren't valid - at least I have found "single point of failure". Because of time constraints I had to fix it with block of ugly code. The project needs change in the process structure to adapt new features in more seamless way.

When you meet that kind of "spaghetti code" there are two known ways of dealing with it:
- pull one strand at a time - clear one flow path - repeat until structure is simple enough
- eat it all at once - analyze requirements and used implementation and rewrite it from start

Reduce structure, simplify, refactor and do not let the code grow into spaghetti again. It will ruin maintainability and extendability of software project in future.

Scatter chart library - moving to Flot

After playing with Open Flash Chart 2 trying to make interactive scatter plot, I've found severe glitches that popped up during prototyping. Library worked quite well for standard line and bar charts. But for my specific scatter chart I realized that:

• I have to use hacks to provide 0,0 origin based XY axes (OK I could just use standard left/bottom axes)
• there are problems with larger scale values (I didn't checked exactly but values in range to 10000 rendered OK, somewhere above it is some kind of limitation)
• problem with grid rendering for values in less than zero range
• OFC can't parse scientific notation values (data number formatting is a must)
• some minor problems that could be solved by chart tuning and hacking ranges etc.
• after playing with OFC stable version I've checked last trunk version that worked much better

I summed all problems and thought about potential extendability problems and tried to move to library based on HTML Canvas object. One of the simplest and most mature projects is Flot. It has all required features, including interaction. The only problem I found was rendering under IE using Excanvas emulation object. Standard stable Flot version doesn't work well under IE8. Charts were drawn although without data points. After checking project issues list, I've found that some problems where resolved by using newer JQuery and Excanvas library. I've tried to run Flot from trunk version, and my IE8 problems were solved. As a extra newer Flot version contains more interesting plugins like navigation. BTW maybe it's time to publish next Flot stable version in downloads section. The only disadvantage of that solution is forcing IE users to accept running Excanvas security policy.

Choosing scatter chart library for web application

So I definitely know that I have to make web based UI for medical diagnostic application. It's not so complicated as I initially though. It is a kind of scatter chart that has to display delivered XY data and use as reference some kind of meta data like description, database id etc. Basic functionality of that module is adding and editing points. It should be somehow interactive - like has clickable points that can fire various events. Graph zooming is also needed.

There are articles with list of interesting solutions: 13 useful graphing solutions, 10 Free Chart Scripts. I reviewed few charting libraries that can be divided into three categories:

• HTML canvas charts
• Java applets
• Flash based charts

There where also other interesting ideas like SVG based interactive chart, or image chart with imagemap and added DHTML tricks, but still without enough support and needed writing more code.

Needed charting solutions should be lightweight so I thought about html canvas drawing, but unfortunately there are serious IE issues - It does not implement such feature, so excanvas ActiveX is used not always successfully (new IE changes are breaking excanvas functionality).

Another good choice from point of customization and extending functionality is Java applet based on JFreeChart, or lighter jchart2d library. Using Java applet was tempting. I made some serious applet work some time ago so I had to make some user experience tests of deploying UI as applet. The large group of users has no technical computer background, and can be confused about accepting prompts for Java installation (Java is not preinstalled with system in here), waiting for runtime download and finally for applet initializing. That may cause serious problems with acceptance tests.

So I tried Open Flash Chart 2- the nice flash based chart library. It has fixed but all basic features I need. Flash player is king of the browsers (large deployments ratio) today. That and smaller than Java's runtime should cause less problems with runtime installation. It has wrapper libraries in most popular programming languages including Java. I have choose third party jofc2 library for quite easy integration with code sample from Open Flash Chart and Wicket example.

Open Flash Chart has enough javascript interface, to dynamically change data, graph style, add tooltips and script onclick events on data points. It has no direct zooming support but can be implemented as changing chart properties by adding external scripted controls. Redrawing after data delivery is fast also.

Near future brings even more choices like Silverlight, improved Applet/JavaFX runtime or adoption of canvas like solutions.

Further reading:
Scatter chart library - moving to Flot
Looking for simple python chart library

The best programming language 2 - Simple case tale

The University needed temporary solution for generating list of quiz questions, chosen randomly from data prepared in structured text file. The scope of application was nothing comparing to "Enterprise Systems", so I treated it as a kind of entertaining enterprise.

The core characteristics were:

• simple logic
• UI forms for getting parameters from user
• printable or saveable formatted questions list
• low burden installation and use (get it on pendrive or download ready to use).
• main constraints: short time and possibly low cost.

I started looking for similar open source products, but not found needed functionality. So when I checked myself for Not Invented Here syndrome, I started thinking about custom solution.

There were many ways of implementation. I thought about preparing windows exe file with included required dependencies (no installing additional Java, .Net, Python or other runtime).
I started playing in "prototyping with Python"- it was fun and things were going smoothly. But I stopped before making UI. Plain old console was "too lame" for final users, so I planned GUI with TkInter included by default in python installation to avoid porting to windows executable problems.

I get core set of functionalities, but user interface still was barely specified. Then I get many additional requests about "last minute" changes in UI forms (and probably not the last). The second "economic" option was to implement the whole application in Java Script and HTML running under web browser. Every computer has web browser suitable to run that kind of application so there are almost no installation and portability issues. Implementation time should be short too. To speed up things I have changed data format from flat file of records into JSON structure. Porting logic was fast too. The most time took me making forms working.

First version was not so fancy and needed some GUI tweaking like formatting, changing labels and headers. In a case of any further UI changes other people knowing HTML could make fixes instantly. The extra feature was possibility of "deploying" the little application on a web server as HTML page. Saving result to file wasn't working directly. Making it really automatic needed interacting with browser security policies for access to local file system. We decided that it will be still usable if user use copy-paste technique, and place formatted result in external text editor. The default usage was printing questions on paper so nobody saw problem there.

Everything is working as designed, and shows that web browser should be treaded as good platform for small applications. Sometimes little prototyping gives better insight on a whole system.

How to get (rid of) Anemic Domain Antipattern coding multitieried enterprisey application

Programmers want to make "good designed", "enterprise like" application. So keeping in mind boxy schema of multitiered application they start to organize code into separate layers like most common DAO,Service and Domain. Very often that way leads to Anemic Domain Antipattern. It's characterized by "dumb" C struct like domain objects without logic in methods.

The mechanism is simple: programmer wants to keep layers separate and dependent only on common interfaces. To reduce wiring and simplify initial development DAO object are connected to Service layer which uses also Domain objects. Service layer is filled by Script like methods, written in procedural style. That way of development is easy at the beginning. So main logic goes to Services "layer" and data operations go to DAO layer. After some time Services become really fat, but domain objects gain only more attributes and accessor methods. As long as application is a kind of a prototype, or just is small everything works quite good.

But to get more benefits of object oriented programming in future, we need net of more "intelligent" domain objects "talking" to each other, not just logic closed in services layer scripts. At this moment that could only be done by total refactoring, or just by designing from scratch using experience from previous implementation. Just keep in mind mantra encapsulation, reusability, simplicity.

The simplest cure to that antipattern is to design "good wiring" at the begging, to give domain object direct access to DAO layer, and other low level services needed by domain logic. Then, domain can evolve easy into more complex structures, needed by bigger "enterprise" applications.

Another point of view on Object Oriented Programming problems

It's not another article about OOP pros and cons.
I want to present you a non orthodox example of dealing with common programming problems without using standard OOP patterns.

The site is for some time on the web already, and presents controversial articles about OOP and SQL criticism.

I don't agree with author's overall point of view, but many ideas are very usable in specific simple applications development, and some of patterns presented there could be found in current RAD frameworks.

Just read and brainstorm.

How to learn simple programming in a fun way

I have looked for games that incorporate programming or learning programming and I found few interesting gems. One is SCRATCH visual programming environment for kids made by MIT with sets of basic flow control and sprite sound and animation commands. I have tried some tricks with LOGO like pen drawing features using drag and drop interface and it looks intuitive and appealing enough to catch curious kids attention.

On the other hand I found less appealing low tech c-jump board game with goal of stepping through C like syntax simple arithmetic program. It is simple but still shows the concept of execution of sequenced commands with flow control.

The direction is promising for Microsoft, as they are developing KODU project, aiming at children too. The concept looks like interface for BPEL kind of language and probably in future that kind of environments will be more popular. You can find there links to more similar projects.

Blending gaming with simple programming may be a catch for working adults too.

The art of exceptions catching

Exceptions in programming languages are very useful tool for propagation information about program errors and helping handling critical situations properly. But it is only tool and how it can be useful depends only on implementing programmer.

I can put it in few points what I have learned in theory and verified during practice:

• If you don't know what to do with exceptions don't catch'em - exceptions reveal errors so fix errors - don't hide them under carpet

• Catch exception if you can handle it, you know how to clean up used resources and how to inform developers/users about that event

• Don't use exceptions mechanism (hacky way) in the correct flow of the program

• In Java use unchecked exceptions (RuntimeExceptions descendants) instead of checked Exceptions - it saves "throws SomeException" litter in code forcefully reminding about catching them (it makes sense only in some kind of low level services with inevitable critical situations that can be handled by direct clients)

I think it's only basic set of rules.

Do you have another ideas for exception handling?

How to export data to one big XML file

My company had to add extra export format for data in a system I'm working on. I get specification and XML schema and started analyzing it for best fast solution. Data was ready to export so there left decision about choosing right way to make XML. Deadline was imminent, and everybody was really busy at work. Making export file - was good task for our newest programmer. He never parsed/created XML using SAX/DOM or other libraries, but was envy to use and learn one of the standard.

The main technological problem was the size of output XML. For biggest data case it would generate over 500MB XML with about 50000 nodes at highest level. It would be efficiently generated using only writing partially to stream technique like SAX. I have chose two options- use SAX or own similar simple API - realization time was crucial. Because XML structure was relatively easy - all tags without attributes, so we decided to use plain string buffer technique with addition of couple helper functions.

At the beginning I had objections about not using standard libraries that should help. But the implementation of XML format moved forwards, output file was easy to validate with XML schema. The most time was committed to handle non existing or incomplete data cases. The export function was ready before time. That is one of examples where using simplest tools, meets desired goal.

What are your choices in similar situation?

Tools for static java code checking

Java language with its strong typing is well suitable for static code analysis.
There is variety of tools for early bugs and flaws detection through automated code checking. I mention three open source tools that any java programmer should give a try:
- PMD
- CheckStyle
- FindBugs

All tools have set of flaw detection rules. Every one has some strengths and weaknesses - all three are complementary to each other. I suggest you to run every of it and look at the different kind of flaws they explore. with default rules. It's possible of course to define own set of rules suitable for your needs. I've tried to install eclipse plugins for those tools - only PMD runs without problems. Maybe I should refresh my Eclipse 3.1 installation. PMD has a "copy and paste" duplication detection tool- but somehow it is not working with some characters in UTF-8 encoding. I hope it would be fixed - it's really useful tool.

So install code checkers - and view reports of your code. You can be surprised. It gives the opportunity to chop off the heads of some simple but nasty bugs.

PyCha chart drawing library working under SPING

I've ported PyCha chart drawing library to work under SPING instead of Cairo. I've discussed that idea with PyCha lead developer - Lorenzo Gil Sanchez.

Lorenzo said he will still stick with Cairo, that is good library ported and available for many systems. Anyway I've ported PyCha to SPING and tested with SPING creating SpingCha off-project.


Here is lastly created PyCha news group , where you can find posts about PyCha development.
And - if anyone is interested - working SpingCha release in files section.

Fighting with duplication - attack of the clones

Last week I proceeded with clean up tasks on one project I'm working now. It's big number of jsp generated reports. Some time ago I setup initial environment for such reports, so even analytics without special JSP or JAVA knowledge could do some reports. It was temporary solution, before jumping to more enterprise tool. Results were not so bad - new reports were added very quick. Everything worked and was usable so enterprise tool is just forgotten mention.

Over two years it overgrowth overall code base and many problem issues and problems were related with those reports. In harder cases I had to dig into that and what I have seen is a massive duplication. I any possible form. So when we found bug in one section of code there was huge probability that the same problem was duplicated more times elsewhere. It was even worse. Instead of new features added to base report there where created new versions of the same report with new parameters or features. And divergence between those reports arises over time. So you can imagine how much time maintenance started cost at some point.

I like to stick a specially with one coding rule: do not create new and remove existing duplication.

Duplication starts when some functional block of code appears at least in two places in code. Sometimes it is just simple one liner - but even then you should consider pros and cons for wrapping that piece of code in some procedural statement.

Let's back to vicious mechanism of avalanche of duplicated code in non programmers environment - when people don't know good "coding" practices. One of reports (let's call it A) was copied with some functional difference (as B). We have got almost two identical files. Then comes another feature that seems to interfering with previous one. Because new feature ticket was assigned only to one of those reports (after some time everybody sees two reports and thinks - those are two separate reports) and new feature seems so "new" and practice before was to create separate functional report (for "clear solution") so there comes new report C derived from A. After some time somebody realized that there is no such feature for B report. Continuing previous process we have now 4 reports with D version. Then comes major change - added new module that shows new values in similar manner so base reports A,B,C,D are copied as 2A,2B,2C,2D and then changed some values and layout. Almost pure copy'n'paste coding.

Having one simple report at the beginning that would have parametrized additional features it's now eight versions of duplicated code with less than 10% differences. Cost of removing bug or implementing new feature is to about eight times bigger than for single but more complicated file. Not including effects of further "extension" mechanism - avalanche just gains new mass in exponential rate.

You could think what kind of procedures you have there to allow that kind of practices. Ok - team of report analytics was told to eliminate visible duplications, but it wasn't enough. When somebody concentrates on complex analytical problem doesn't think much about removing duplication. Even when report is complex and amount of code and sql is enormous. It's just additional burden that seems not helpful for analytical problem solving.

What could help? Some quick course of coding practices and techniques in context of that environment - basics how to write reusable pieces of code, avoid common pitfalls and reduce codebase to ease development and maintenance. And more practical explanatory examples. Sometimes duplication problem is visible but there is no simple solution - then pairing with more experienced programmer would help. It's just organizational issue.

So now the team is fighting with reports bugs and feature requests. And the biggest impact we are gaining now by not touching those new features or bugs but by merging and removing duplicates. It's ironic but number of bugs is decreasing now in reverse exponential rate.

Thought of a day- unplanted easter eggs

The funniest easter eggs you can find sometimes browsing source code with distance. There are unintentional pearls of some kind of evil genius.

Rolling my own simple python chart library

Two weeks ago I started looking for simple python chart library. There are interesting solutions based on cairo, GD and alike libraries. I was surprised there is no simple chart drawing library using "core" python installation environment. I found of course nice working PyCha using cairo, but there were problems on one testing servers with cairo library installation.

Portability is one of my priorities so I've been trying to find more flexible solution. So I've found SPING (Simple Platform INdependent Graphics former now as PIDDLE). It supports numerous backends like PDF, PIL, QuickDraw, PostScript, Illustrator, piddleVCR, OpenGL, Tk, wxWindows and support for others is in development. Looks promising. Ok so it gives me basic drawing capabilities. What about beatiful charts? After online research, I decided that something like PyCha would be enough. But PyCha uses cairo ... . Then idea has come- rewrite PyCha to use SPING. Idea is not so stupid- PyCha library is quite small and porting should be doable.

So I started rewriting PyCha fo SPING (codename SpingCha). The biggest part of work is porting calls typical for cairo interface to more generic SPING. I used Interface pattern with simple cairo like API. After two days I have almost working implementation, tested on original PyCha examples. By saying almost - I mean it needs better color, fonts and antialiasing handling to produce more beautiful charts.

After finishing and cleaning up code I'm planning to publish it under derived GNU Lesser General Public License. Maybe I will talk with author of PyCha to just include my ideas in main project. Meanwhile if you are interested to get SpingCha just add your comment, I will know if I need to speed up things, and release it.

UPDATE: About further story you can read new post

web.py update 3 (and tough bugs)

My new application on web.py framework works great. I eliminated slowness issue during installation of new web.py environment.
It was caused by opening db connection without pooling- and it took really long time (about 10 s). So next time I properly installed DBUtils. Run tests and found exception raised by web.py database. I had to make quick fix in my web.py installation - changed name of one of parameters in DBUtils function call.

Few days ago I tested web.py session support and authentication in my application. It worked good, when something broke next day. I had feeling that session isn't properly stored or recalled- and my authorization module redirected me to login page. Access logs showed me that actually I requested only login page, not any other. Strange. I was looking for bug in web.py, but there had to be another reason I thought at end.
I used another web browser- in my case IE. Login and session worked well, better than under Firefox. The reason causing problem was performance enhancing plugin for firefox I installed few days ago to tune in new Firefox. It used strange caching algorithm, and every time I hit some application page login page was loaded. Probably it was first page cached by plugin. I uninstalled it and reviewed other browser modifications.

Once again experience shows that the most time consuming bugs are simple and just hidden in area assumed for sure as free of bugs (in my case I was looking in server app, but bug was caused by browser client modification).

Happy programmer's day!

Today (September 13th) is the 256th day of the year celebrated as unnoficial "programer's day" holiday.
It looks like IT community is developing its own culture and holidays. We have system specific internal calendars based on "epoch day", important historical events like y2k, and many more that probably you can find.
So if you are programmer or just like the idea, celebrate it and have a fun!

The best programming language - choose the right tool for the job.

Is there the only one ultimate programming language, what is the best match for every application?

Some of folks may say- "Yes, It's L**. I've used it for X years in production environments and there is nothing in the world better than that". But it's not yet another thread about java vs c++ vs python vs ruby vs php [vs pick_your_language]...

Look at the current situation of existing programming languages. There are probably more programming languages than ever before and they are still evolving, what is IMHO good thing. An evolutionary criterion here is mainly market demand. It regulates need for that kind of tools and the most obsolete ones are vanishing.

Most of the programming languages start existence as a project of small group of enthusiasts or scientists. One person driven projects are quite usual. When it gains wider audience as good candidate for programming environment there is a moment when market influence kicks in.
From business point of view software production process must be optimized to get needed set of features, using possibly smallest resources and time to get final product of desired quality.
So there is economical pressure to keep (man time cost)/(product quality) ratio low.

What is programming environment? The main element is of course programming language concept with its constructs, and a set of basic framework libraries. It must be implemented, so it goes with tools like compilers, interpreters, virtual machines, debuggers and other preparing software to run and monitor software in target system. On the other end there are Integrated Development Environments boosting programmers’ productivity. All of these elements are used during software production process which has strategic impact on software cost and quality. The total cost and quality depends on the combination of lower level facilities of programming environments (I am excluding people skills).

All programming environments have some sweet points there and sometimes hard to reach or lack of other features somewhere else. So there is no one simple and effective tool for creating software.

So how to choose best one for your needs? Productivity depends on domain and size of application, architecture, speed of elementary analyze-write-compile-test cycle, easiness to create small and clean code, good debug information, matching tools power to programmer skills and other minor factors. These factors should be evaluated during short "evaluation runs" to get the complete picture.

How to collect such valuable metrics? We need productivity measuring features seamlessly hooked up into existing tools used during software development and maintenance. Having recorded software feature production time metrics and process structure can give us very valuable information. It gives possibility to look back, find weak points of process and try to improve it. Or, you would try to compare metrics in context of features and application character for different programming languages and choose the best for your needs. But we need tools to acquire those metrics. Some of commercial tools are going into that direction, and I think it's time to incorporate that into platforms supporting creation of open source software and sharing information with wider audience.

There is a very long way (if any exists) to one leading programming environment with simple but powerful programming language and well designed set of libraries etc. We have different competing programming environments find their market niches today. Evaluating and selection of best solution isn't trivial task. But if you can - think about it and choose the right tool for the job.

Maybe one day there will be one ideal small and complete programming language with good framework library, with scripting for small programs and support for big enterprise application features, with changeable features and many sets of well designed libraries and frameworks. Sounds like utopia for me, but who knows?