|
Monday, June 11, 2007
As a Java/J2EE person most of the time working in development probably deals with using IDE’s and less on command lines. But I will express my love for command line tools here since it makes things faster and useful.
IDE’s are good but has its share of limitations:
a) Processing log files when running application server: Log files generate different statements and may be you would like to know all instances when a particular event happened. Following Unix command will bring all lines which contains Exception instead of having to go thru the file looking for it.
> cat server.log|grep Exception
b) Want to automate the build: If we have used command line options for builds using tools like Ant - its just one step away in QA environment. No separate effort for QA
c) Multiple steps thru IDE are error prone: Ex: suppose before we start testing the application we would like to copy some files to the server. Every-time we do it manually or using features like linking folders is not preferred since human steps are error prone and the development directory gets fixed to a given location if we use linking.
d) Suppose before every check-in you want to test your code automatically: From Ant build scripts we can keep a target which will refresh the database, perform unit tests for all the service classes.
e) Single enter is easier than multiple clicks: when you have to get a job done in a UI using multiple clicks and if the same has been put in s script – all you have to do is press enter. Easy – isn’t it !!
As a command line interface Unix shells brings a lot of tools to perform jobs quickly:
a) Need to find which process is running in a given port.
>Fport | grep 8080
Fport is a windows tool which tells you which process is running at different ports.
b) Need to copy a file to a Unix server running SSH server. No ftp server is running for security reasons.
Ø scp
(Win-scp is one utility which allows one to do so)
c) Schedule a job: crontab – very nice tools in Unix and for Windows there is ‘at’ command to do this job.
d) Few more tools which perform better file management and transfer and without having to download and configure additional tools are: Tar, zip etc.
The key features which makes it powerful over dos shells is its capability of using the output of one command as the input to other and which allows someone to perform an operation by using pipes and achieving a complex operation is a small script.
Editors in Unix:
Well probably I wont refer to Unix editors for editing the java files although its much quicker to work with it. People when gets used to it don’t wanna quit but for initial users its less intuitive than a Visual IDE. Plus IDE has support for debugging etc. The main thing is one does not have to take his hand out of the keyboard while working with IDE mean working with mouse and the keyboard and going back and forth between it. So simple editing I will still use VI but for java programming Intellij IDEA/Eclipse.
UnixShell in Windows:
For unix lovers there are several options to simulate unix shells in windows. Options are
a) Cygwin
b) MKS toolkit
c) Interix
Shell scripting:
Moment someone starts working on application maintenance the need to log file processing to understand and track-down a defect or outage becomes even more prominent. Shell scripts to add logic and the single line commands to programmatic option add a lot of value. The key file processing and parsing utilities are awk and sed.
There are scripting languages like Perl also helps to perform some of these operations really well.
To conclude:
What is engineering?
Engineering means that the truth is present – i.e. we know how to solve a problem and there are possibly many ways to do it. Engineering will means using the right tool for the right purpose and every engineer needs to know more than one tool to perform the job. The good engineer knows more tools – i.e. for a java engineer knowing just java is not good enough since its not going to solve all the problems. A single line of Unix command can take 500 lines of Java code. Knowing the tools and applying the right tool when you need it makes all the difference.
What is science ?
Science is about finding truth. Most of the times all we do is engineering thus should be in constant lookout for tools which makes our life easier.
References:
http://en.wikipedia.org/wiki/Sed
http://en.wikipedia.org/wiki/AWK_%28programming_language%29
http://en.wikipedia.org/wiki/Crontab
Thursday, May 24, 2007
Understanding HTTP requests:
From a browser when a request gets made the first point it hits is the DNS server where the name is resolved and the IP address is retrieved. Once this is done then IP address resolution mechanism is used to track down the server which is hosting the application. The protocol being used is HTTP which has different parts like header, request method names (GET Vs POST) etc.
Ex:
GET /index.html HTTP/1.1
Host: www.example.com
Response:
HTTP/1.1 200 OK
Date: Mon, 23 May 2005 22:38:34 GMT
Server: Apache/1.3.27 (Unix) (Red-Hat/Linux)
Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT
Etag: "3f80f-1b6-3e1cb03b"
Accept-Ranges: bytes
Content-Length: 438
Connection: close
Content-Type: text/html; charset=UTF-8
HTTP is a stateless protocol.
What is meant by Stateless protocol?
A stateless protocol means that in this protocol when a caller makes a request and gets a response back, server do no remember who the person was. Next time the same person makes another request then it will be treated a fresh request.
But in many applications we will need the user to maintain state. Ex: User logs in, enters the user id and password. Then he visits some more pages but does not want to enter the username in all of those pages. Then the server will need to maintain the state details of the user like username and then use it for rest of the pages.
Whats the solution?
The common method for solving this problem involves the use of sending and requesting cookies. Other methods include server side sessions, hidden variables (when current page is a form), and URL encoded parameters (such as index.php?userid=3).
Server side sessions:
In J2EE this is called Http-Session. Session data is actually stored in the memory space of the web server. There are different ways session data can be stored and depends on the web server/application server implementation.
i) In Memory: This means the session will be stored in the memory space of the Web Server/App server Process.
ii) Storing session in databases.
In some cases there are services (another process) which manages the session persistence.
This means the way to store is not part of any spec or standards and is part.
Now when a request comes in from a client browser to the server cookie which contains the sessionId gets passed to the server which then server decodes and uses it for getting the session data. Session cookies are transient in nature, i.e. not stored in file system as such but can be seen using certain tools. These cookies are set to expire (be deleted) upon closing the browser. Cookies that last beyond a user's session (i.e., "Remember Me" option) are termed "persistent" cookies. Persistent cookies are usually stored on the user's hard drive. Their location is determined according to the particular operating system and browser (e.g., C:\Documents and Settings\username\Cookies for Internet Explorer on Windows 2000).
Cookie content looks like this:
JSESSIONID=1607EB0EF22CAAA1FC82056B4978F40D
In cases where browser reject cookies then one can configure the application server to use url-rewriting where URL contains the session id embedded in there.
This is from Xpflow and shows how session id is generated in the browser memory space. Cookievalue=admin@eforceglobal.com is the persistent cookie.
How different application servers maintain sessions?
Jrun: http://www.adobe.com/cfusion/knowledgebase/index.cfm?id=tn_18131
BEA: http://edocs.bea.com/wls/docs90/webapp/sessions.html
WAS: http://www.phptr.com/articles/article.asp?p=332851&seqNum=2&rl=1
Session replication & Clusters:
What are clusters?
For high availability we group a set of application servers where each of them is capable to handling user requests. Depending on load any given server can be used at any given point of time.
Failover & Session replication:
Now when a given server goes down then to the user it should not appear that something has gone wrong. The session data should move over to other server which can take the request processing from that point onwards. Thus in order for failover to take place session replication needs to be done – which means session data needs to be shared with other servers. If the session is stored in-memory then this needs to copied to other servers in the cluster and if its in the database then its automatically supports shared session data.
Session binding listeners:
This addresses the problem when we would like to take some action based on the event that session is created to expired. Ex: Every-time a user session gets expired we would like to perform some cleanup of the resources and user may not have clicked on Logoff button but rather just closed the window or has not accessed the system for more than session time out period. In these cases session binding come handy.
Session Hijacking will mean someone gets hold of the sessionId and the fakes the request and perform operations to the application.
Reference:
http://www.webopedia.com/TERM/S/session_cookie.html
http://java.sun.com/developer/EJTechTips/2003/tt0626.html
Analyze browser cookies: http://www.sandsprite.com/Sleuth/overviews/Sleuth1.4_overview.html
http://www.imperva.com/application_defense_center/glossary/session_hijacking.html
http://en.wikipedia.org/wiki/Session_hijacking
Thursday, May 17, 2007
For stress testing RIA based few options are present. Now when the application is developed in Flex it is not very different from a JSP or ASP applications with respect to how it sends the data back to the server. The protocols can be HTTP(s), AMF(s) or RTMP(s). When the application is developed using Flex HttpServices then the problem of load testing becomes rather easier since its the same protocol being used by a Jsp/Ajax/Asp application. Thus options open source/free like Jmeter works fine.
Jmeter experience:
Jmeter provides two ways to record the interactions with the borwser and we can go ahead - increase the virtual users and generate more load like we do for Load runner etc - but this is open source. Using Badboy - its a very easy, intuitive tool to do the recording. One can record the scripts and then export to JMX format.
However here is the catch, this was tried to capture the HTTP traffic coming from a flex UI and this fails to recognize the requests. Not sure why - seems like a bug where it expects only HTML pages.
Next option is trying out the browser proxy which Jmeter allows one to do. This is quite neat and works. We tried with Mozilla but works for other browsers as well. This is in a way good, i.e. one less interface to learn - same way we navigate the application a browser we can very well go ahead and use it. After starting the Proxy server one will have to go thru the application and Jmeter will record the steps. The main difference from Badboy is, there is no visual replay mode where one can see how the wholo pages are navigated.
Blocks:
None of the steps after login works. Says session invalid. This needs a deeper understanding of how Jmeter works and how session management works in J2EE applications. Since the session management was beig done thru cookie thus cookie was not being captured and sent with every request till some cookie manager related settings were done.
This article is very useful and helped thru plus Jmeter help documentation.
Reference:
http://nerds-central.blogspot.com/2006/08/pushing-envelope-with-jmeter.html
Thursday, March 15, 2007
Flex is the application building framework from Adobe which adds a lot of value in developing application for Rich UI and better user experience. Lets begin with Rich Internet application development.
What is Rich UI?
UI looking like Windows application, Swing application with widgets like Tabbed pane etc. HTML does not provide any standard tags for these complex widgets. There are frameworks – server-side and client side which does the trick.
What are the RIA options present?
If we look into the space of RIA frameworks different names will come into pictures – AJAX frameworks, JSF, Open Lazlo, Flex, Applets.
AJAX frameworks: AJAX frameworks are java-script based frameworks – which uses some java script functions plus some Java script libraries to develop Rich UI. People with HTML/JS skill-set can work with this easily. Cross browser support for all browsers is not present in all the libraries.
JSF: Sun has put a lot thrust behind this so that this becomes a front end framework. Its server-side – which means tag library driven and its HTML. There are frameworks like Jboss seam – which tries to put together UI framework and back-end frameworks together to provide seamless integration between layers.
JSF in action: http://www.irian.at/myfaces/home.jsf
XUL – XML based UI development for Mozilla and Firefox. Do not seem fully supported by IE. IE7 has support of XAML which is Microsoft version of XUL.
Applets and applet based frameworks: There are a few Applets based RIA options like Canoo (http://www.canoo.com/ulc/). Although Applet has features to build RIA applications its biggest disadvantage continues to be the limited availability of the current version of the technology in Web browsers. Lack of Microsoft support is also part of the reason. Compared to Flex the arguments will be XML based UI development in Flex is an easier programming model than purely java code driven layout design.
Java-script Vs Flash?
Why java and .NET became popular – since its platform independent- i.e. the cost of software development goes down. Same reason applies for flash since it provides a programming model for developing web application tier which is browser independent – thus the development and testing becomes less time-consuming for developing cross browser compliant applications. For all browsers Flash players are available which allows one to develop Web applications quickly – since we testing time is less and less error prone programming model where users don’t have to choice of using browser specific java script or CSS in the first place.
Open Lazlo: Flash based. The programming model is similar to Flex (MXML/Action-script) Vs XML-Java-script. It can generate RIA in Flash as well as DHTML.
Couple of arguments which holds against Open Lazlo compared to Flex:
i) IDE support needs improvement.
ii) Flex is backed by Adobe instead of a group of developers.
iii) Flex is enterprise application framework instead of UI only framework.
Is Flex a RIA framework?
The answer is Yes and NO. Yes since it supports RIA development and No because it is much more than that. It has a set of features under data-services where it provides features like server push, polling which adds a lot of values to enterprise applications.
Is Flex free?
For RIA application development - the SDK is free. Yes, if you want to use your own editors and command line features for compiling the MXML to flash then its free.
- The Flex SDK which is free and includes the Flex libraries, the compiler (mxmlc), the debugger, and the documentation.
- Flex Data Services (FDS), an optional set of server-side components deployed in your J2EE application server. FDS includes a Java RPC service (sample 3), publish/subscribe messaging (samples 6 and 7), and data management services (sample 8). FDS is free for a single-CPU deployment (FDS Express), and is licensed per CPU when deployed on multiple CPUs.
- FlexBuilder, an optional IDE for Flex development. Built as an Eclipse plug-in, FlexBuilder includes a design view and a code view, code hinting, visual debugging, etc. FlexBuilder is licensed on a per developer basis.
- Optional charting components licensed on a per developer basis.
We can develop and deploy Flex applications entirely for free using the SDK and the IDE of our choice.
However Flex builder is what developers should adopt – for quick design, preview and compilation, debug and bunch of other features which is present. Not expensive ~ 1000$. We have licenses to be used.
The server-side components for Flex data services (in multi-CPU mode) are comparatively expensive and we have licenses for this as well.
Learning Flex: How difficult is it?
Flex is rather new but concepts are not. People with Action script or Java backgrounds with object oriented concepts seem to pick up Flex well. I hear some people say that “Client server days are back” - i.e. there is a good degree of similarity between Client server programming with VB, or Swing applications and Flex UI widgets, containers etc.
Gottcha’s
Is it all golden? Well – its not.
Few browser/platform compatibility issues: We have come across a few differences in behaviors between Safari in Mac and IE - Windows. I will say that’s just one instance. Other than that it was pretty smooth sailing as far as testing goes across browsers.
Support of Test Automation tools:
Limited. Only with QTP which is a commercial software. A lot of the tools don’t recognize the Flex objects. Few open source tools are there but quite primitive in nature.
How to work with QTP 9.1 and Flex 2.0.1?
Please note, earlier versions of Flex is not supported by QTP. Only 9.1 supports Flex automation. For QTP 9.1 and Flex 2.0.1 – here is the steps
http://weblogs.macromedia.com/lin/archives/2007/01/tips_for_config.cfm
Workaround: If not planning to buy or use trial versions for QTP - develop unit test cases for business tier. The UI widgets from Flex are well tested – and thus regression are more expected to be there in the Business Logic tier and for end to end testing we can manage by using manual testing.
Support of Performance engineering tools:
Limited. Only one which works is Silk performer – which is commercial software.
Open source tools do not work for all scenarios – reason Flex allows to communicate with back-end over many protocols - RTMP(s), AMF(s) and HTTP(s). Out of this Server push is done over RTMP and most of tools support HTTP primarily. Silk Performer supports HTTP and AMF.
Adobe lab has a tool for load testing flex applications which tests for data services. http://labs.adobe.com/wiki/index.php/Flex_Stress_Testing_Framework
As most load testing tool vendors have not yet released support for RTMP(s) or the latest version of AMF(s), the Flex Data Services Stress Testing Framework allows Flex developers to load test their applications that use RTMP and AMF3 without needing to use a commercial testing tool.
Different tool vendors are working towards adding support for Flex/Flash applications and expected to release in near future.
Workarounds: Open source tools like Jmeter provides ways for load testing web services. http://dev2dev.bea.com/pub/a/2006/08/jmeter-performance-testing.html
Thus we can expose the middle tier services as Web services and do load testing at the back end tier. Most of the UI is residing in client and don’t add a whole lot of performance overhead. So overall application performance will be – backend performance plus data transfer time.
I have AJAX enabled application – can I use it with Flex?
Flex can be adopted in pieces – Ex: If we have a real time monitoring system needed in an application we can just use Flex for that – no need to re-architect the entire application.
Yes; http://labs.adobe.com/wiki/index.php/Flex-Ajax_Bridge
What happens if I get stuck during development – can I Google and find the answer?
The community is fast growing. There are forums which help to answer questions other than a pretty well documented help documents. I would say there will be some cases we will need to rely on documentation vs quick solutions from the forums. There are code cookbooks which are being growing so overall help is there and growing.
Forums:
FlexCoder Yahoo group
Cflex.net forum
Adobe Forum on Flex – found some useful responsibility.
Flex resources: (Thanks DR)
http://www.flex.org/
http://labs.adobe.com/flexproductline/ (there is a lot of info in the community links on this page in the flex forums and MXNA aggregator links)
http://www.cflex.net/
http://weblogs.macromedia.com/mc/
http://labs.adobe.com/wiki/index.php/
http://www.iterationtwo.com/frameworks.html
http://video.google.com/videoplay?docid=4762944544960468107 (Kevin Lynch's Web 2.0 Conference Presentation announcing Flex 2.0 and Flash 8.5 - Google Video)
A link from a Flex engineer on RIA design considerations
http://kuwamoto.org/2005/06/08/seven-ideas-for-building-better-rias/
These demos with source code samples may provide ideas for some of UI portions of the DAM and approvals modules that will provide views of asset and product thumbnails
http://www.quietlyscheming.com/blog/components/fisheye-component/
http://www.quietlyscheming.com/blog/2006/05/09/animated-dragtile-component-01/
With the formal release of Flex 2 Adobe has launched www.flex.org as an entry point for flex resources and blogs from the engineering team. There are also archived breeze sessions from on line seminars this month at http://www.onflex.org/ted/2006/06/adobe-developer-week-sessions.php including one on FDS Flex Data Services and AS 3: ActionScript 3 for Flash Developers.
Here is an article on FDS at http://www.adobe.com/devnet/flex/articles/architecting_rias.html and some FDS messaging related posts at
http://www.onflex.org/ted/2006/05/flex-data-services-part-1.php
http://www.onflex.org/ted/2006/06/flex-data-services-part-2-messaging.php
Flex components available for downloading at the Adobe exchange
http://www.adobe.com/cfusion/exchange/index.cfm?extid=1047291&view=sn610#loc=en_us&view=sn100&viewName=Adobe Exchange&extid=1047291
The newly released auto complete component looks like it could be used on form input fields:
http://weblogs.macromedia.com/flexteam/
Click below for more:
http://www.adobe.com/cfusion/exchange/index.cfm?extid=1047291&view=sn610#loc=en_us&view=sn611&viewName=Flex Extension&authorid=73322349&page=0&scrollPos=0&subcatid=0&snid=sn611&itemnumber=0&extid=1047291&catid=0
These are some of the types of nice to have features with image views.
http://www.richapps.de/?p=38"
Can be explored as time permits.
http://mxdj.sys-con.com/read/255823.htm">Here</a>
Here are some more articles on mixing Flash and Flex that may be useful when constructing full client in Flex
Example of using Flash for the login component:
http://flexblog.faratasystems.com/?p=88
Presentation at http://www.jessewarden.com/archives/2006/08/flash_flex_powe.html.
Video link at: http://www.jessewarden.com/archives/2006/09/flex_seminar_pr.html as well as interesting blog reading on Flash designing and Flex programming
http://www.amitgupta.info/E41ST/RIA/E41ST.html">http://www.amitgupta.info/E41ST/RIA/E41ST.html</a>
Enter as guest and use a keyword such as "clinton" to find images of books that match the keyword.
The use of the close button for the details screen which appears after clicking an image allows for more room to view the images.
This demo has the tree component with checkboxes for multiple selection. View the demo at http://www.darronschall.com/weblog/archives/000241.cfm
http://adobedev.adobe.acrobat.com/p71169528/
This presentation is long (57 minutes) but has good tips for designing and constructing Flex apps for responsiveness.
Note on slide 14 discusses a Flickr app that caused redundant measurement layout cycles for each image object.
Designing Flex 2 skins:
http://www.adobe.com/devnet/flex/articles/flex_skins.html
Tuesday, March 13, 2007
Recently started using this – although it has been there for a while. Pretty neat service – People bookmark the pages and this information is shared among people to search better.
http://del.icio.us/
How does it help?
Example suppose you are looking for useful information on a topic say JBPM etc. What are the options - Google off-courseJ. Now how do we get pre-rated pages i.e. if someone has found it useful then he can rate the page as how important it is. With any search engine we depend on the relevance ranking algorithms of the search engine - which may be simple like number of times the word has occurred or much complex algorithms or formulas. Although this mechanism is good (PageRank for Google) it isn’t smart - not as good as someone reading the page rating it - since till date there is no standard way of publishing information in a web page - i.e. so that web crawlers or software agents/programs can read and extract the information from a page. This is what we call semantic web http://en.wikipedia.org/wiki/Semantic_Web. Web 3.0 is also heading towards it where agents can read, parse and understand the pages better - kind of Applied AI in web. But till the time we have semantic web or even if we have it in place, we would like other web surfers are finding useful and that’s where delicious comes into picture. This is a simple solution to solve a common problem. We can see how many people have bookmarked it and gives some idea on the popularity of the page as well.
How can you help others?
Rate the page using the downloaded browser plugin and attach proper tags. This way collaboratively everyone benefits.
Reference: Google page ranking (http://www.google.com/technology/)
Thursday, March 01, 2007
Rational Rose being one of older tools for modeling purposes has advantage like going back and forth between class diagram and sequence diagrams. As far as detailed design goes one of the issues which is faced is managing the Rose models in a team environment, i.e. if there is more that one person working on the model at the same time. When people work on the same model and prepare two different versions of it, this gets difficult to merge them into one. The option of Copy any paste from one model to the other is not an option – since only one model can be kept open at any given point of time (As of Rational Rose 2000).
The other problem that happens when people work in detailed design on different modules or use cases of the same project and yet we have a need to put them all together since end of the day its one product. There will be some areas which is common to all modules for those manual merging has to be done but overall we can break down the overall UML artifacts in modules and we should be able to put them together using Controlled Unit feature of Rational Rose.
http://www.aka.org.cn/_Others/se_chatter/study/teamwork/using_case_tools.html
Models and controlled units
When a Rose model first is created it is stored in a model file with the extension .mdl. Rose uses a straightforward ascii format called petal for these model files. As the model grows, and also to allow several people to work on the same model, the model can be broken up into one or more controlled units. (This is analogous to how for example a C++ program is broken up to be stored across multiple files.) These controlled units are created to define work units which individual developers can work on and forms the configuration items you can put under version control. Controlled units can be created out of packages in a model. To create a new controlled unit, simply select a package in a diagram and do File:Units:Control.... You can also do Browse:Units (F7) which will show the potential controlled units you can create. When you create a controlled unit, you will be asked for a file name to use to store this new controlled unit 1. Note that packages in the Logical and the Use Case views are stored with extension .cat, packages in the component view are stored with extension .sub. In addition, you can also make the deployment diagrams with it's processors and nodes a controlled unit (.prc), as well as the model code generation property sets (.prp)
When you have created a controlled unit, the .mdl file will no longer store the contents of that package, it will now be stored in the new controlled unit. The model file will now reference this controlled unit (similar, but not the same, as a C++ #include). Next time you open the .mdl file, Rose will ask you if you also want to load the attached controlled units 2.
Took this as the starting point and went ahead merging different models for different releases.
Merge the models:
The first problem which had to be solved was there were models developed by different authors which needed to be put together.
Step 1:
Thus the first thing which was done is classes, class diagrams and sequence diagrams for each module were kept in a package. Once a package is created then the next step will converting each package to a controlled unit. A controlled unit becomes a embeddable component.
Step 2:
Save the unit as a category file with .cat extension.
Thus from different models – category files were generated which holds class, class diagrams and sequence diagrams and when put into one model – necessary copying of items can be done.
The same strategy when followed allows someone to work on one or more units and when done loaded into the central model.
Limitations and Gottchas:
a) When controlling a unit which includes sequence diagrams the classes also needs to be part of the package.
b) If an existing package exists with the same name as the package being imported from an unit – then quite intuitively that generates an error.
What about exporting the rose model? When should I use it?
Exporting Rose model to Petal files seems to be a way to store Rose models so that it can be exported and imported from other tools.
Reference:
a) http://www.comp.utas.edu.au/documentation/rational_rose/Rose_TeamDev.pdf
b) http://www.aka.org.cn/_Others/se_chatter/study/teamwork/using_case_tools.html
|