Convert from HTML to XML with HTML Tidy

For few days I was involved with WSO2 Mashup Server 2.0 release documentation, giving a hand to the mashup team. Documentation is a painful task, but when comes to open source what matters mostly is documentation :D.
Last night I had to convert a bunch of html files (some Java Api Docs) to xml in-order to port into maven site. Formatting 30+ html files to xml !@#$%^&*@% :D. So I was googleing for a tool to automate the task. With few clicks here and there I found a nice article in Big Blue‘s developer works site, a tool called “Tidy“. When I tried to download and use I figure out that you can straight away apt-get the package and use. So,

sudo apt-get install tidy

and your box is now equiped with the tool, and can be accessed via the shell.

tidy -asxhtml -numeric < index.html > index.xml

but who wants to convert file by file when you have such a nice tool, so I spent few minutes in writing a tiny shell script to get the job done, the snippet is,

#!/bin/bash
for file in $(find $1 -type f -iname '*.html'); do
	myf=`echo $file | sed 's/html/xml/g'`
	tidy -asxhtml -numeric < $file > $myf
done

All looked good, worked fine. However in my Api Docs I had, had few special tags, custom to our Mashup Apis (<imconfig>, <yahoo>, <mail:config>). Tidy gave error for these files since the tags are not recognized.

In such a case you can train Tidy for new tags, by adding few lines to the tidy configuration file. (/etc/tidy.config – You can also give your own config file at the prompt)

new-pre-tags: imconfig, yahoo, msn, aim, icq, jabber, username, password

There are whole bunch of tweeks you can do with tidy, [1], [2] and [3] are some useful links that you can read up when using the tool.

[1] : http://www.ibm.com/developerworks/library/x-tiptidy.html
[2] : http://tidy.sourceforge.net/
[3] : http://tidy.sourceforge.net/docs/tidy_man.html

Lesson learnt the hard way

Terminator was running in four segments, top left was a maven build, top right was a svn update, in one of other two an apt-get was running and I was playing on the smaller one left in bottom right. My notebook was running on battery power and I forgot to plug it in to AC, before the box got hibernate. Once I reboot it with AC power my X-Server was gone !! I couldn’t see the log-in screen and it just left me with the console in full screen.

I am not a linux guru, but i tried few commands, “startx / dkpg -reconfigure / nvidia-config / etc”. I had to run to our sys admin since he is considered the guru :D. with few commands at flash speed he got the x-server running again. but it was all broken here and there. couldn’t hear sound, installed apps were not working and above all when I reboot, I had to type “startx” each time to see the display (Which was a pain and made me itchy :P). Chamith’s idea was even if he could get the x-server running in normal mode, some errors that am getting wont be able to fix. so the best option to make all smooth is to re install the OS and he said “No probs ne machan (sinhala way of saying dude / bro ;) ), ur home is in a different partition right ? so its a matter of ten minutes to restore the root in the other partition.” – OH CRAP NOOO my /home was also in the same partition and I only had one single big partition. :(

SO yeah I had no other option left, I had to back up 20GB of my /home dir and re-install ubuntu 9.04. (Coping 20GB from here and there is a crappy and a messy job) I was struggling with the fix till today morning and finally got everything smooth again.

SO the lesson from all of this is. keep your /home mounted in a different partition. I learnt the lesson, the hard way.

SL vs NZ T20 | My 1st time among international cricket audience

Its like this, I am not a huge cricket fan, the matches make me bored along the game. One guy balls, another guy slowly hits and 8 people try to send him to bench (but should mention, when the batsmen hits hard or when others send him to bench its always fun :D ). Being in Sri Lanka one thing you inherit is the cricket fever, from your grand dad to siblings, they all watch (or maybe play) cricket. The Wills world cup (1996), I just rem from here and there, (I was in grade 5 :D good times) is one nice memory I have, Sri Lanka winning the big cup and we all were staring at the TV with great joy.

Anyways with all that background story, yesterday I happened to be there at R. Premadasa international cricket stadium to watch the Sri Lanka vs New Zealand T20 match. Folks at work made plans to watch the match for weeks and finally 12 turned up. (This time the numbers were low coz there was a rumor that SL lost the SL vs Pakistan match coz they were there that day) however I took a ticket (Dumindu Threatened me to buy :D) and was there few minutes before the game started.

Dilshan on fire :D

New Zealand batted 1st and the game became fun, with all the cheering, shouting and screaming. The foreigners were good they played 141 for 8 wickets and it was the time for the home players. I was amused, I was not bored in fact I enjoyed it so much and was waiting till our innings start.

WSO2 Cricket fans WSO2 Cricket fans

The SL inning started after about half an hour with some drops of rain, but it was all good. The fans were shouting.. “Go SRI LANKA”, “Common SANGA / SANA / DILSHAN” and I was among them too. The 1st half of the innings was great, Dilshan played well with a 50 in 23 balls lifting the SL run rate. My hopes were for Jayasuriya coz he was my hero since 96, but he was out with 1 run and it was disappointing. With Sri Lanka’s 3rd wicket (Sangakara) the batsmen kept falling. and there was no one to carry on Dilshan’s run rate. but until the last over we had the chance to win, it was 9 runs with 7 balls, then 8 with 5 and finally 3 with 1. So we lost. :(

I saw the faces of people around me, all their happy faces are turned dark, and it was sad indeed. But well for me I was happy, (Not for we lost) I enjoyed the match, the time I spent, even the defeat, we gave a good fight. So yeah it was one great experience for a newbie cricket fan.

Writing custom queries to retrieve data from WSO2 Governance Registry

WSO2 Governance Registry is a big part of wso2 governance product stack. Even though it is primarily aimed at managing, versioning, rating, and commenting on SOA artifacts it can also be used as a simple data store. with the 3.0 version the G-Reg gave support to custom query execution from the client side. This feature helps immensely when you use the registry for non-standard tasks. For me I had to do some pagination work for the comments that belongs to a particular resource, hence my approach was to write few custom quires to get the job done. The code is as follows.

	/**
	 * Returns a chunk of comments
	 * 
	 * @param resPath	Path to the comment
	 * @param start		The beginning index
	 * @param size		Size of the chunk
	 * @return			an array of comments
	 */
 public Comment[] getCommentSet(String resPath, int start, int size) {
		Registry registry = null;
		try {
			registry = ; // get an instance of the registry 

			Resource comQuery = registry.newResource();

                        // The Sql Statement
			String sql = "SELECT REG_COMMENT_ID FROM REG_RESOURCE_COMMENT RC, REG_RESOURCE R, REG_PATH P WHERE "
					+ "RC.REG_VERSION=R.REG_VERSION AND "
					+ "R.REG_NAME=? AND "
					+ "P.REG_PATH_VALUE=? AND "
					+ "P.REG_PATH_ID=R.REG_PATH_ID LIMIT ?, ?";

			// Set SQL statement as the resource content
                        comQuery.setContent(sql);

                       // Setting the media type and properties
			comQuery.setMediaType(RegistryConstants.SQL_QUERY_MEDIA_TYPE);
			comQuery.addProperty(RegistryConstants.RESULT_TYPE_PROPERTY_NAME, RegistryConstants.COMMENTS_RESULT_TYPE);

			registry.put("system/myQueries/query", comQuery);
                 String resourceName = "testResource";
                 String pathToResource = "/system/myResources"

			Map params = new HashMap();

                        //Setting the parameters
			params.put("1", resourceName);
			params.put("2", pathToResource);
			params.put("3", start);
			params.put("4", size);

                       // Executing the SQL statement
			Collection qResults = registry.executeQuery("system/myQueries/query", params);

			String[] qPaths = (String[]) qResults.getContent();

			Comment[] comments = new Comment[qPaths.length];
                        // Loading the comment data to comment object array 
			for (int i = 0; i < qPaths.length; i++) {
				if (registry.resourceExists(qPaths[i])) {
					comments[i] = (Comment) registry.get(qPaths[i]);
				}
			}

			return comments;

		} catch (Exception e) {
			String errorMsg = "Backend server error - could not get comment set";
			log.error(new MyTestException(errorMsg, e));
			return null;
		}

	}

Yeah simple as that you get your resources set without much effort. A big thank goes to Dimuthu

Best of SOA Summer School

During this entire summer, WSO2 conducted a series of trainings on SOA best practices, How-Tos and trends, titled “SOA Summer School“. With the dawn of the autumn the series of online summer classes came to an end. With the popular demand, a 3 hour session is going to be held tomorrow the 13th August summarizing the 8 weeks of lessons and highlighting their key areas.



Untitled

for a better insight I have attached tomorrow’s agenda, and it is conducted by Samisa Abeysinghe, Ruwan Linton, Prabath Siriwardena, Keith Chapman and Nandika Jayawardena. The online trainings are totally free and any one who is interested in SOA and EA can gain a good knowledge of its dynamics.