Good practices for backups on DVD

DVDs are good and cheap media for middle term backups of old files, or periodical snapshots that fit into 4.37 GiB space. Best example of mine is storing old photos.

Choose you disc

Choose good quality discs from known brand/manufacturer relying on reliability statistics

Choose DVD+R over DVD-R - +R uses better writing method and has better error correction, check this article for long explanation. Do not use rewritable discs that use erasable data layer, that could be more easily damaged.

Burning speed - the lower the best - higher speed means potentially more errors to correct during writing and reading. Most hardcore safe backup setting will be x1 , but 4x or 8x should be quite safe when today DVD writers give you x16 writing speed.

Handling

DVDs are fragile to physical damages, specially vulnerable to scratches. So discs should be handled with care, without touching recorded surface. Discs surface should be clean to avoid scratching by hard dust particles. Blank discs should be perfect clean before writing, as any speck on surface will block laser beam.

Do not place adhesive labels and use special CD/DVD markers for labeling. The best place for tiny label is not recordable small middle ring. Why is that? Label side of CD or DVD is separated only by thin layer, that chemicals could easy get through and damage data layer with recorded pits.

Recorder DVDs should be stored in dark and dry place. Here is nice list of DVD handling recommendations on a NIST page.

Additional backup safety measures

Redundancy increases probability of data retrieval. For critical data it makes sense to make more copies. Other copy could be placed off-site, for better protection in fire and flood proof place.

Another option would be additional error correction data created by ECC software like dvdisaster. The best option is to write additional ECC data scattered on the same disc with data. It's a trade-off of disc capacity for additional data safety.

Every media has limited life span even stored in best conditions. Some manufacturers give even 30 years (or even 100 years!) for DVD discs, but I'm not such optimistic. Stored archives should be periodically checked for errors (dvdisaster has that functionality), and then moved to new media. It makes also sense in case of technology change and migration. Today next popular optical format is Blu-Ray Disc, but its still young.

Software development project size and methodology

Here is a short description of software development characteristics for projects of different size and its impact for methodology.

Very simple application (single programmer, tester - one man for all)

Requirements are simple and all known. If you are writing very simple application, using good architecture plan and code writing practices, as a result you should have clean ready to change application. Adding automated tests makes changes easier without software quality drop. You know all details without looking into documentation, and testing new changes and features during implementation. All changes go to the code repository with comments and TODO file.

Simple/medium, application (team of 1+ programmers and other stakeholders)

The requirements should be gathered and agreed before development. You are using some kind of Issue Management System, tracing all change requests. Application is divided into main modules referred in task details. Programmers know what to change and what are dependencies between modules and side effects of implemented changes. Testers know main modules, and are going through test procedures of modules and functionalities. Procedures are short and clean and there is a natural place for agile methodologies.

Large application, (team of 10+ programmers and stakeholders)

Application is quite huge. Preparing requirements is often separate project. Every change request is thoroughly reviewed then approved. Application have many interdependencies caused by re-usability and connection points. Some changes have strictly local effects for given feature, those are less harmful. Other change some lower level service, probably unit tested, but still having impact on bigger process. Automated tests are not covering every possible use of code, manual top level functional tests are last line of defense in quality assurance. In real life every module has many connections with other modules. Testers have very limited knowledge about those dependencies. Everyone needs more plans, documents, procedures and artifacts that is slowing down the whole process. Practical solution is to divide the large project into smaller, less complex sub-projects.

At this level of complexity there are many "process complete" management methodologies. The winning one is the best suitable for given project, reducing risks and allowing to finish project at planned costs and time.

Plan for data backup and recovery

I've started writing about data backup and recovery as a necessity in a "digital era". To make it simple and successful it needs some upfront planning.
I've brought together most important issues about safe data backup and recovery. No matter if you have you PC hard disk full of pictures or have lots of business data on workstation, you should consider following subjects:

• Why do you need to make backups? (read Backup your important data article)


• Categorize your data - easy manageable big chunks divided by type/importance


• Plan for recovery - what is saving data worth if you are unable getting it back?


• Consider data retention - time horizon (short vs long term backups)


• Choose storage media/place - what suits your needs


• Simplify and automate backup procedure - use specialized software or scripts to keep your backup plan going with minimum work from your side


• Monitor condition and manage backup copies - think about change if process doesn't meet your needs

Maybe it seems dull and hard to implement, but everyone should cut it to their needs.
It's all about time and costs, but having saved your data is sometimes priceless.

Database related general application performance tips

Most of the serious applications nowadays use some kind of relational database engine. Database becomes main data source and data processing module in application architecture. That's the reason why it's common application performance bottleneck and main scalability factor.

Having design and data flow requirements its possible to prepare database system at the system planning and implementation phase. Don't get to much into performance details at that stage because "premature optimization is the root of the evil". Sometimes system requirements and design meets reality and final expectations, but changes in requirements that could change application data flow are more common. The most accurate results are coming from real data and users usage patterns, so getting system responsiveness metrics is crucial even at the early prototype stages.

Having such data about performance "weak points" is a start point to optimize and improve overall system scalability. It's good to begin from top more abstract layers in application architecture before rushing to lower level database storage tweaking.

There are some tips divided by subsystem scopes and ordered by top to bottom abstraction level:

Application scope

• discuss with project stakeholders responsiveness requirements for various application functionalities
• analyze data usage patterns like writes vs reads, common data usage vs casual reporting, written once or changed often etc.- it gives image what could be improved, and what kind of underlying database mechanism you will need
• remove worst bottleneck (having biggest performance impact) first to get best results
• use cache for most used data (reads and writes if its possible)
• design transactions smart- long transactions cause performance problems
• at first you should use normalized data schema, but there are situations where little denormalization is crucial for good performance (f.e. copying some data to most read table to eliminate need for joining other big tables)

Database system scope

• use indexes where it works best (every index adds performance penalty for data writes)
• use vertical and horizontal data partitioning - move less used data into other tables or use database engine specific features
• configure database and use its features like special table types, special indexes for your best

Operating system scope:

• use database dedicated host or performance cluster - for large scale systems
• check network latency and throughput for large data transmissions
• tune underlying disks structure and file system- separate partitions or disks for database files, use low overhead file system or custom database "raw" partitions (like in Oracle DB)

The meaning of "realistic" in FPS games

Grog News wrote long and interesting article about reality of guns, gears and tactics in FPS games. Even games most rated as "realistic" have large discrepancies vs reality.

Although gaming industry considers fun more important over realistic features, new FPS games are incorporating more complicated graphics, physics and AI engines, to raise playing experience to the next level.

Games having good balanced reality and fun, are tough enough to keep interest of players for a long time (like Counter Strike type games). Too much realistic constraints in game are resulting in too hard and annoying experience (f.e. player can't jump 20 meters down, or can't run long distance keeping constant high speed). It's an entertainment industry so usually more fun means more profit.

I have to admit that even simple change in details pointed in Grog News article, could improve both fun and reality. It doesn't even had to be some kind of new fancy physics engine. Let's take for example weapons and ammunition/magazine compatibility issues or bunny jumping tactic to avoid shots. I really would like to have features like ricochets effects, but there is a field to improve simple mechanisms first.

There is a bright future for "reality simulating" games, these are just slow evolving.

Backup your important data

Organizations are collecting more and more data on their workstations/servers. They are doing also regular data backups to prevent loss of assets, time and money. Are you doing data backups? Are you ready to lose your data?

Simplest hard disk drive failures are bad-blocks affecting single files. If file is important there is good chance that it can be recovered using proprietary tool. More destructive could be malware erasing or damaging random files, or user hitting "delete all" button on wrong marked files.

Hard disk drives aren't perfect and there is quite large probability that you hard disk will die. The last resort then are data recovery companies, but the cost starts from about thousand of dollars and more.

What about laptop theft or lost external drive? There are events that could physicaly destroy your hardware, like fire f.e. Then probably all is gone.

Backup your important data. In best case you will save your time or money. In worst you will prevent of loss of valuable data.

Little change in the blog title

Choosing good blog title could be difficult art. I have chosen "IT Pro Life" that seemed good so far. Maybe it is too pompous and enigmatic, but I wanted something simple and short, that defines character of that site.

When I started analyzing keywords I realized that many times my content is in "pro-life" context. It seems people may be confused finding totally unrelated content
("what? pro-life? It looks like no life!").
My bad, I didn't seen that at the start.

I'm doing little change in title to IT Pr0 Life, using slang word for "professional" that is: pr0 (PR zero). It will look more funny, but there is place for sense of humor too :)

I keep domain name untouched, I will see how it works with search engines.