awk still rocks!

Last week I faced a unique challenging problem where I need to extract multiline soap request from a huge log file. First I thought about using java, then realised that loading a huge file is java could be problematic.  Then I thought about groovy or other interpreted language. Then lastly I found a very simple solution based on awk. I did some awk script in past, for generating huge test file based on some random criteria(Like generating CDR files for load testing). I was simply amazed to see how an awk script can solve this problem so easily.

Lets assume you have huge file with following SOAP request.

   Payload: <?xml version="1.0" encoding="UTF-8"?><vivek xmlns="com.test.vivek">
<Service>
	<Template>abc service template</Template>
	<XRef/>
	<LogMessage/>
	<CallBackUrl/>
	<Delivery>
		<Synchronous/>
	</Delivery>
</Service>
<Model>
	<Head>
		<From>test@test.com</From>
		<Subject>Technical Service &amp; Repair</Subject>
	</Head>
	<ContactUs>
		<Sender>
			<FirstName1>vivek</FirstName1>
			<FirstName2>vivek</FirstName2>
			<LastName1>kumar</LastName1>
			<LastName2>kumar</LastName2>
			<PhoneNumber>9674174848</PhoneNumber>
		</Sender>
	</ContactUs>
</Model>
</vivek>

Here is the the awk script to extract all such xml:

awk '/vivek/,/\/vivek>/'  test.txt   | awk '/Payload:/ {print "*****************"; print } !/Payload/{ print;} ' 

This awk script will extract all text from vivek to /vivek> then we run another awk script which check for work Payload and if its found then it print “***************” then rest of the line containing Payload text, if line does not contain Payload then it simply print the line. We are printing “**************” to keep a boundary between different matched request.

Here is the response from command:-

awk.png

Awk Reference

 

 

Kindle screen freeze issue

I have been using this Kindle 3 from last 8 years. I found this device very useful considering it has a very long battery life and portable. When I was reading last, I found battery to be critically low and left the device. After couple of days when I checked it, for my utter surprise its screen froze, I tried to reset from reset button(hold start button for 20 seconds), but it did not worked.

First I was heisted to open it, considering I was hopeful that it might be a charging issue and keep charging the device continually. But I did not turn up. Then one fine day, I thought as its already dead(or say gone), there is no harm if I open it up and see.
   I searched on google and found reference of a reset point, where I can shot it with a wire to restart the device. I keep trying with different option and finally it worked. It’s highlighted in below screen.
   Note that there are some version of kindle which already has a reset button(highlighted below, first image). in those type it is just a matter of pressing the reset button.
   In my case, I had no such button, the challenge was to figure out two points which needs to be shorted to reset the device. Finally I used a safety pin to short these and and it all started working.

Update after a long time..

Much time has passed since I write my last post. lots of  important things happened since I  had written my last last post, most important of those is moving to new organization and new city after a long time. I had some doubts and fear before moving to AKQA, but now I feel like its worth moving to new younger company.

Another thing, is to learn new stuff, CQ or(AEM) and whole lot of new technical movement that is  happening around in DevOps  (Chef, puppet) Microservices, which is the driving factor around all..

Here I found one interesting video about Microservices

Last but not least and most important

<

New Work

My friends has started a new site

Node Internationalizatio(I18N) approach

While searching for some work on I18N implementation approach on Node, I came across following three approach.

  • User library, example i18Next library. Example
  • Using html/javascript translation with html 5 data- attributes. Example. Here top language header is added dynamically. And translation is done using javascript node value manipulation.
  • Using different subdomain. Example

    .

  • ArrayList for loop puzzle

    Let me start this entry by asking a question. What would be the output of following two programs?
    Program1:-

    import java.util.*;
    public class ClassA{
            public static void main(String args[]){
             List<String> list = new ArrayList<String>();
             list.add("1");
             list.add("2");
             for(String x: list){
              list.remove("1");
             }
            System.out.println(list);
       }
    }
    

    Program2:

    import java.util.*;
    public class ClassA{
            public static void main(String args[]){
             List<String> list = new ArrayList<String>();
                             list.add("1");
             list.add("2");
             list.add("3");
            for(String x: list){
              list.remove("1");
             }
            System.out.println(list);
    }
    }
    

    Output:

    Program 1:

    [2]


    Program 2:-

    Exception in thread "main" java.util.ConcurrentModificationException
            at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
            at java.util.AbstractList$Itr.next(AbstractList.java:343)
            at ClassA.main(ClassA.java:8)
    

    Description:-

    Program 1 will run without any issue and delete “1” from the list and hence the output will print list content with having one element “2”. Whereas program 2 throws ConcurrentModificationException.
    Why?
    Here is the explanation.
    Java for each loop internally uses Iterable and Iterator to loop through each element of the collection. When we are looping through the foreach loop, we implicitly call list.iterator(), list.hasNext() and list.next() methods. In case of ArrayList list.iterator() returns instance of ArrayList.Itr class. This instance has a expectedModCount varable which keep track of current state of list. We can assume it to be the stamp of current state of the list. Whenever we modify the list(add/update/delete element) this modCount variable gets changes. When AbstractList.Itr is instantiated it initialized expectedModCount with current modCount of the list.(Fir understanding purpose we can say that Itr has variable which keep track of current state of the underlying list.
    When we delete an element modCount is increased and when next time itr.next() is called Itr.next() matches the mod count and throws ConcurrentModificationException if it does not match.

    The whole idea is to stop someone from modifying the underlying collection once we get Iterator and working on it.

    Then comes last question? Why no error was throws in case of Program1. It is because there are no element to be loop over and no Itr.next() method is called and hence no ConcurrentModificationException.

    Itr.remove() keeps the consistency and reassign expectedModCount.

    ArrayList source code:- http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/ArrayList.java

    atan2 function problem in db2,java,javascript

    Last day I was working on a formula to calculate the bearing(and finally direction from point 1 to point 2). I was using the formula described here. I was running same formula in java as well as DB2. But getting different result. After much analysis I found the problem with the atan2 API in DB2 and Java.

    Java API accept y and x as first and second argument where as DB2 API accepts in reverse order.  I also found that  java script also accept y argument first. This is quite bad. I don’t know which one is right but everyone should follow same pattern. This difference in API was the source of the bug.

    During this analysis I also found that Db2 client(squirrel) was not returning the precise value where as Db2 client was returning. sqirrel_db2