November 8, 2008

.NET Thread Pool Extension

Recently I have finished one of the most exciting component designs over the past couple of months. The customer wanted to have an extention to the .NET ThreadPool class, which would provide him with the control over individual worker threads. Although by default ThreadPool does not provide access to the individual background threads which it use, Stephen Toub wrote a couple of articles at MSDN magazine with an explanation how to do that (as an example you may take a look at the article here )

The other requirement of the design was to select jobs for execution based on priority and to change the priority of the jobs in the runtime. For the former task we could use the Priority Queue implementation. The latter task was challanging since that Priority Queue implementation had only two methods - Enqueue and Dequeue. So there was no possibility to look through the Queue to change the priority of the item. To solve this problem I used the following strategy:
  1. A wrapper class was provided for the job, which was identified by a unique ID
  2. The user could use the ID of the job to indicate that the priority for this job should be changed
  3. The Thread Pool class used an internal job ID and a mapping from the user's ID to the internal one
  4. The Priority Queue contained the internal job IDs
  5. When the user wanted to change the priority of the job the following iterations took place:

    • New internal ID for the job was created and sent to the Priority Queue
    • Mapping from the user's ID to the internal one was updated
    • The system remembered that the old internal ID is not valid any more and should not be processed when it will be
      dequeued


http://www.linkedin.com/in/oldbam

October 15, 2008

Software Engineering Radio - Ebay Architecture Principles

Recently I have discovered a site with fabulous podcasts dedicated to various questions in software engineering, Software Engineering Radio (http://se-radio.net/)

There was an episode on Ebay architecture principles, some of the items from which I decided to rewrite here.

  • Partition everything. "If you can't split it - you can't scale it"

  • Functional segmentation. The selling system is distinct from the bidding system and from the buying system. Each system is supported by separate DBs.

  • Horizontal splitting. Each functional segment is chopped into smaller pieces. (e.g. if key ends in 1 - the first database is employed, if key ends in 2 - the second database is used, and so on.) On Ebay it is done in a way similar to Hibernate Shards.

  • No distributed transactions are used.
    The Details table is updated before the Master table, if the first fails, the second won't be updated; if the second fails, the asynchronous event will later recover the database from the inconsistent state.

    If the Item DB and the User DB should be updated, the importance of having the first one in consistent state is more urgent, so only the Item DB is updated and then asynchronous event is generated to update the User DB later.

  • No session state is stored on the server. For multi-pages functionality the following approaches are used: URL rewriting, cookies (all the state information is put in the cookie), writing current session data to the scratch database (for multi-pages flows)

  • Application features can be switched on and off without code changes
    The code deployment is different from feature deployment.
    Features can be checked for the availability by the code.
The whole podcast is available at this link



http://www.linkedin.com/in/oldbam

September 28, 2008

Eclipse UML plugins

Eclipse (http://www.eclipse.org/) is a freeware well-known open source IDE. A number of plugins were created, which turn this IDE into a powerful tool. The two freeware UML plugins, which I use with Eclipse are the following:

  1. AmaterasUML
    Very simple tool. You have to update diagrams manually if the source code was
    changed (no reverse engineering). However, it supports import from source files to UML diagrams and has a cool automatic diagram layout feature.

  2. Relo - Relationship based Exploration
    A tool for large projects, which allows to visualize relationships between
    classes. They have a cool 2min 30sec screencast on the plugin site, which shows how to use this tool (worth seeing)





http://www.linkedin.com/in/oldbam

September 16, 2008

Opinion Mining and Sentiment Analysis

I have finished a two-month internship in Oslo at Medallia Nordic. The aim of the project was to create a Text Analysis module, which could retrieve opinion phrases from the user reviews. Probably the best set of articles about the topic is contained at this page: http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html


http://www.linkedin.com/in/oldbam

June 17, 2008

We lost the Android Developer Challenge

With the team of students from Hasso-Plattner Institute (Potsdam, Germany) I developed an application for Android Developer Challenge. Our intention was to make a travel guide for people, by combining the Wikipedia articles with the Google Maps. Some screenshots are available below






The development of the application was done within the framework of Tempus project ITSoftTeam. Thanks to Artem Polyvyanyy, Andriy Vedrych and Sergei Smirnov for their efforts in this competition.


http://www.linkedin.com/in/oldbam

March 1, 2008

Implementing a Menu with Submenus in a C console application


Recently, I had to implement a menu in a C console application, where items where grouped in submenus. What I wanted to have was a possibility to see the same submenu items after I have finished executing some submenu code. To clarify this idea (as my poor english might produce a lot of mess) I will ask you to look at the below picture.



As you see, when I select "Menu 2", I see specific submenu items. After I call "Menu 2 : Submenu - 1" and the specific subroutines executes, I see the items of the first submenu. To return to the "Main menu" I need to select "Exit".


Now we will look at code. First of all we need to define constants for menu items.

const int MENU_MAIN        = 1;
const int MENU_1 = 2;
const int MENU_1_SUBMENU_1 = 3;
const int MENU_2 = 4;
const int MENU_2_SUBMENU_1 = 5;
const int MENU_2_SUBMENU_2 = 6;
const int MENU_EXIT = 7;

The main() function will call the displayMenu helper function and then execute subroutines depending on the user choice.

void main()
{
int menuOption = MENU_MAIN;
do {
menuOption = displayMenu(menuOption);
switch(menuOption) {
case MENU_1_SUBMENU_1:
do_menu_1_submenu_1_function();
menuOption = MENU_1;
break;
case MENU_2_SUBMENU_1:
do_menu_2_submenu_1_function();
menuOption = MENU_2;
break;
case MENU_2_SUBMENU_2:
do_menu_2_submenu_2_function();
menuOption = MENU_2;
break;
}
} while (menuOption != MENU_EXIT);
}


Let's discuss the above code a bit. At first we initialize menuOption variable to show the "Main menu". Then we call the displayMenu function, which returns user input. The subtle thing is the reassigning menuOption after executing some routines based on the user choice. This lets us show the same submenu items from which we selected our subroutine.

Now we have to show the function, which accepts user input.


int displayMenu(int startMenu)
{
int userChoice = -1;
switch(startMenu) {
case MENU_MAIN:
// display main menu
printf("-- Main Menu --\n");
printf( "1. Menu - 1\n2. Menu - 2\n3. Exit\n\n" );
while( userChoice<1>3 ) {
printf( "Please choose an option (1-3): " );
scanf( "%d", &userChoice );
}
// process user choice
switch(userChoice) {
case 1: return displayMenu(MENU_1);
case 2: return displayMenu(MENU_2);
case 3: return MENU_EXIT;
}
// we should never come to this point
assert(0);
case MENU_1:
// display submenu 1
printf( "\t-- Menu 1 --\n");
printf( "\t1. Menu 1 : SubMenu - 1\n\t2. Exit\n\n" );
while( userChoice<1>2 ) {
printf( "\tPlease choose an option (1-2): " );
scanf( "%d", &userChoice );
}
// process user choice
switch(userChoice) {
case 1: return MENU_1_SUBMENU_1;
case 2: return MENU_MAIN;
}
// we should never come to this point
assert(0);
case MENU_2:
// similar to case MENU_1
}
}


At the beginning the user sees "Main menu". When he selects some submenu, the displayMenu function makes a recursive call with a specific start submenu:


// process user choice
switch(userChoice) {
case 1: return displayMenu(MENU_1);
case 2: return displayMenu(MENU_2);
case 3: return MENU_EXIT;


As you see, the displayMenu function can only return one of the following values: {MENU_1_SUBMENU_1, MENU_2_SUBMENU_1, MENU_2_SUBMENU_2, MENU_EXIT }. Actually, all the other values, like MENU_MAIN, MENU_1 and MENU_2 denote only the current manu layer (i.e. top menu or submenus) and do not result in executing some specific business logic. That is why in the main() function we get from the displayMenu() functions only these values, which carry meaningful user input (i.e. MENU_1_SUBMENU_1, MENU_2_SUBMENU_1, MENU_2_SUBMENU_2, MENU_EXIT).


With this technique you can have any number of layered submenus easily; and all the mess is hidden in the displayMenu function :-).

The complete source code for this example can be downloaded here



http://www.linkedin.com/in/oldbam

January 20, 2008

Surfing the University sites, I have found a project, which I did with two my friends during the Methods of OOP course. That was really exciting to find something you've done a year ago and it is working :-)

It's interesting that the idea of this project arised when I looked at the Playboy site :-[ There were a lot of nice pictures there, which I wanted to download, but some of them were really small. So, I decided to make a programm, which would
  • download only certain kind of content (e.g. only pictures)
  • have a number of download filters (e.g. only pictures, which are larger than 400*400)
  • remember files, which were already downloaded, to save internet traffic
Since I had to do a team project for one of the university courses in C++, I decided to find some teammates and implement this in C++.

Frankly speaking, we didn't implemented all the functionality which I wanted. But at least we made a working console application, which receives an URL, load all the data from this URL, parses the links up to the configurable DEPTH and loads all the pages and all the data from these pages in C++ with the help of libcurl and boost. Not bad for a student project, yeah? 8-).

You can download the result here or look at source code here.

P.S. Thanks for my teammates Nazar Grabovskyy and Dmitriy Krasikov ;-)
P.P.S. It would be nice to finish this application sometimes :-)