Archive for category java

String Concatenation optimization

String concatenation is one of the most common, but, a pretty expensive operation. It can hit the performance severly if not used correctly. The performance goes down drastically if you append strings using ‘&’ OR ListAppend() in a loop. I have seen application performance improving by 50-100% just by optimizing String concatenation (though that depends on how much concatenation is used in the app). So what do you about it?
The simplest and the most optimized way to do these append operations is using java’s StringBuffer. (I am sure you must be aware of it but still.. :) ) .
The code would look like

<cfset sb = createObject("java", "java.lang.StringBuffer")>
<cfloop from=1 to=100 index=i>
  <cfset sb.append("something")>
  <cfset sb.append(i)>
</cfloop>
<cfset result=sb.toString()>

Sometimes I feel that we should have a datastructure like this in ColdFusion directly but again I think whats wrong with using StringBuffer? Its like any other function which we would create. Isn’t it so?

If you are a puristic and don’t want to use any java API inside your CF app, there is another simple way to do the same thing. It uses ColdFusion Array to do the same thing what StringBuffer does. Instead of appending the string in the buffer, you can append to the array using ArrayAppend() and then once you are done and want to get the string back, use ArrayToList() with empty string (”") as delimiter. The code would look like

<cfset arr = ArrayNew(1)>
<cfloop from=1 to=100 index=i>
  <cfset ArrayAppend(arr, "something")>
  <cfset ArrayAppend(arr, i)>
</cfloop>
<cfset result=ArrayToList(arr,"")>

This would give a much better performance as compared to concatenation using ‘&’ or using ListAppend() but will have lower performance as compared to StringBuffer. That is because of the overhead of Array object creation and array append operation. ArrayToList() will anyway create the string buffer and append the strings

You should use ‘&’ or ListAppend() only when there are only 2-3 strings to be concatenated. Otherwise always use either of the two techniques above.

Optimizations with literals

Look at these two pieces of code carefully. Is there any difference between these two apart from the fact that the second one is shorter?

<cfset x = "sun,mercury,venus,earth,mars,jupiter,saturn,uranus,pluto,neptune">
<cfset y = ListSort(x,"text")>

and

<cfset y = ListSort("sun,mercury,venus,earth,mars,jupiter,saturn,uranus,pluto,neptune","text")>

If you think there is not much, read on.

There is a huge difference between these two piece of code – in terms of performance. The second one will have much better performance as compared to the first one. How?? Because the ListSort() method in the second case will not even be executed in the page request. Still scratching your head?

It is because of the intelligence that is built into CFML compiler (really superb code written by Edwin Smith). During compilation, it analyzes all the code and wherever there is a literal or functions executing literals, it tries to optimize it. In the second piece here, it sees that ListSort method is being called on a literal and it can be done statically. So compiler will execute this call during compilation itself and set the sorted value on ‘y’. During the page execution, only thing that will get executed will be an assignment. Smart.. isn’t it? Even java compiler, which does quite a many compile time optimization for java source files, does not have this intelligence of executing calls at compile time :)

It is not only about ‘ListSort’. This is true for most of those CF functions which can work on a literal and can return a literal.

Extend CF native Objects – Harnessing Java

Since Coldfusion native objects are java objects, you can harness the java APIs to extend the functionality of these objects. In this post we will take CF Array and see how we can use these APIs to get some cool functionalities from them.

ColdFusion array is actually an implementation of java list (java.util.List). So all the list methods are actually available for Array.
CF provides most of the list functionality using Array functions but there are few things possible with java list which you can not do directly with CF functions.

1. Merge : Lets say you create two arrays and you want to merge these two arrays to create one bigger array. There is no CF function to do this.
However You can call List.addAll() methods to do it.

Here is how it would look.

I am creating array this way just because it is easier and I don’t have to write whole lot of code :)

<cfset y = ListToArray("rupesh,tom,damon,hemant,ashwin,ram,prank,sanjeev")>
<cfset z = ListToArray("dean,manju,jason,tim")>
<cfset y.addAll(z)>
<cfdump var="#y#">

2. Merge in middle : Lets say you want to add the second array somewhere in the middle of first array say after 4 elements. The code would look like

<cfset y = ListToArray("rupesh,tom,damon,hemant,ashwin,ram,prank,sanjeev")>
<cfset z = ListToArray("dean,manju,jason,tim")>
<cfset y.addAll(4, z)>
<cfdump var="#y#">

3. Search : I have heard people complaining that there is no find method in Array. Actually you had it all the time. Just that it was hidden :)
You can use List.Contains() or List.indexOf() methods to achieve that. Here is the code.

<cfset y = ListToArray("rupesh,tom,damon,hemant,ashwin,ram,prank,sanjeev")>
<cfoutput>Contains Hemant: #y.contains("hemant")#</cfoutput>
<cfoutput>Index of damon : #y.indexof("damon")#</cfoutput>

Please note that the java index starts at 0 where as CF index starts at 1. So the index here will be 2. You must also note that since java is case sensitive, this search will also be case sensitive. To build case insensitiveness, you will have to make the list as well as the search in same case – either uppercase it or lowercase it.

4. Search whole array : You can also search if all the elements of one array are present in another array using containsAll() method of java list.

<cfset y = ListToArray("rupesh,tom,damon,hemant,ashwin,ram,prank,sanjeev")>
<cfset z = ListToArray("ram,prank,rupesh")>
 
<cfoutput> y Contains z: #y.containsAll(z)#</cfoutput>

5. Equality check : You can find out if two arrays are same or not using list’s equals method.

<cfset y = ListToArray("rupesh,tom,damon,hemant,ashwin,ram,prank,sanjeev")>
<cfset z = ListToArray("dean, manju,jason,tim")>
<cfset x = ListToArray("rupesh,tom,damon,hemant,ashwin,ram,prank,sanjeev")>
<cfoutput>x equals y : #x.equals(y)#</cfoutput>
<cfoutput>x equals z : #x.equals(z)#</cfoutput>

6. RemoveAll : You can remove all the elements of one array from the second array using removeAll()

<cfset y = ListToArray("rupesh,tom,damon,hemant,ashwin,ram,prank,sanjeev")>
before removal <cfdump var="#y#">
 
<cfset z = ListToArray("ram,prank,tom")>
<cfset y.removeAll(z)>
After removal <cfdump var="#y#">

Go ahead and play around with it!

Connecting to URL from behind a proxy server

I needed a way to set the proxy information on URL/URLConnection and I could not find any good way. One simple way that java recommends is to set the information as system property. These properties are “http.proxyHost” and “http.proxyPort”.

So it can either be set as jvm arguments like

-DproxySet=true -Dhttp.proxyHost=proxyIP -DproxyPort=port

or set them in the code using System.setProperty()

However since this is a system property, it gets set on the VM itself and hence it is not dynamic. In ColdFusion, since tags like ‘cfhttp’ keep them dynamic, I wanted a similar behaviour. After looking around for a while, I noticed that this capability was added in Java 1.5 aka Tiger release. (Is it only me? I keep hitting things which I feel is lacking in java API and then I find them added in 1.5 :) )

In Java 1.5, you can use

URLConnection conn = url.openConnection(proxy); // added in 1.5

where proxy is an object of java.net.Proxy. Pretty neat.

However this was not a solution for me as we are still developing on JDK1.4 (need to consider all the application servers that we have to support). I stumbled upon an interesting article by Daniel Horn who faced the exact same problem. And guess what he did? Since you send an HTTP request to the proxy and then proxy sends out the actual request, he created the URL object by passing proxyHost and proxyPort as IP and port and then he gave the target url string as ‘file’ argument. This is what he did.

String actualUrl = “http://www.adobe.com”;
URL url = new URL(”http”, proxyHost, proxyPort, actualUrl);
URLConnection conn = url.openConnection();
..
..

And it works ! Brilliant !! I wonder why this is not documented in the java API.

Host name verification in HTTPS

As I mentioned in my blog on Missing images in CFDocument, in case you are using HTTPS, you must ensure that the certicate host name matches the host name in the URL. Lets say your certificate is issued to “www.mysite.com” then the request URL must have the host as “www.mysite.com”. It can not be accessed using ‘localhost’, ‘127.0.0.1′, that machine’s IP address or machine’s name.
Till JDK1.3, Sun’s SSL implementation never used to verify the host name of the certificate. Since JDK1.4, it now verifies the hostname to prevent URL spoofing (When I request for some URL, some other guy in between intercepts and sends his own certificate and I will remain under the impression that I was getting the certificate of the requested server and hence a threat).

In case you want to access the URL using localhost or IP address or using machine’s name, there is a workaround possible but that would invlove wrting some java code.
HttpsURLConnection that is used to make the connection, uses an interface HostnameVerifier to verify the host name of the certificate. A default implementation is used by default. You can provide your own implementation of this interface and set it on HttpsURLConnection. That will give the control of host name verification in your hand and you can verifiy it the way you want.

Unfortunately, this interface and HttpsURLConnection are present in both javax.net.ssl and com.sun.net.ssl package. So depending on which SSL packages are being used, you will have to implement appropriate interface and you will have to set this on appropriate HttpsURLConnection. To be sure, let your implementation class implement both the interface and set it on both the HttpsURLConnection by calling the static method

setDefaultHostnameVerifier(HostnameVerifier)

A simplistic implementation which disables any host name verification could be like

    class MyHostnameVerifier implements com.sun.net.ssl.HostnameVerifier, javax.net.ssl.HostnameVerifier{        public boolean verify(String urlHostName, String certHostName){            return true;        }

        public boolean verify(String urlHost, SSLSession sslSession){            return true;        }    }

set this verifier to both HttpsURLConnection at appropriate place.

    MyHostnameVerifier verifier = new MyHostnameVerifier();    javax.net.ssl.HttpsURLConnection.setDefaultHostnameVerifier(verifier);    com.sun.net.ssl.HttpsURLConnection.setDefaultHostnameVerifier(verifier);

Thread dumps…

Many a times our customers complain of Coldfusion Server(CF) hanging, becoming non-responsive, taking 100% cpu or some request taking too much time. In order to understand the reasons why this happens, only way is to analyze the “Thread dumps”. Not many of our CF users are java geeks and therefore when we ask them to send a thread dump, they have a very little clue about it. I hope they wouldn’t be clueless anymore ;)

What exactly is “Thread dump”
“Thread dump” basically gives you information on what each of the thread in the VM is doing at any given point of time. This makes “Thread dump” an excellent debugging tool. It can tell you the states of each of the thread in the VM, where exactly each thread is in the execution path at that point, which thread is waiting and where is it waiting and lot more. It also shows you the stack of each thread and can help you track the execution path.

How do I take a Thread dump?
For any Java VM, if you are running it from a command prompt, you can get the thread dump by pressing Ctrl+Break (for windows) or Ctrl+\ (for unix machines) on the terminal running the server.
In case you are running the VM in background or as services, you can send a “SIGQUIT” signal to the process to get the thread dump. To send this signal to VM on unix machine, you can use “kill -SIGQUIT [pid]” where ‘pid’ is the process id of the server process. This will send the signal to the VM to dump the thread stack on the error stream (On Sun JVM) or to a new file (IBM’s JVM).

You can also use the Stacktrace tool to get the thread dump. It is a nice java webstart application that you can run on the machine running the server. All you need is to specify the “process id” and it will nicely show the threaddump in its own window.

To be more specific for CF, if the standalone CF server or CF on J2EE server has a terminal, use Ctrl+Break or Ctrl+\ depending on the platform. If its running as background process or as services, use Stacktrace tool I mentioned earlier.

Here is how the thread dump would look.

Full thread dump Java HotSpot(TM) Server VM (1.5.0_04-b05 mixed mode):

"web-4" prio=5 tid=0x28999418 nid=0x1460 runnable [0x2fbaf000..0x2fbafd18]       at java.io.WinNTFileSystem.canonicalize0(Native Method)       at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:374)       at java.io.File.getCanonicalPath(File.java:531)       at java.io.FilePermission$1.run(FilePermission.java:218)       at java.security.AccessController.doPrivileged(Native Method)       at java.io.FilePermission.init(FilePermission.java:212)       at java.io.FilePermission.(FilePermission.java:264)       at java.lang.SecurityManager.checkRead(SecurityManager.java:871)       at java.io.File.exists(File.java:700)       at jrunx.resource.FileResource.exists(FileResource.java:98)       at jrunx.resource.FileResource.getURL(FileResource.java:140)       at jrun.servlet.JRunServletContext.getResource(JRunServletContext.java:192)       at jrun.servlet.file.FileServlet.service(FileServlet.java:148)       at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:91)       at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)       at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:257)       at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:541)       at jrun.servlet.http.WebService.invokeRunnable(WebService.java:172)       at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:318)       at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:426)       at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:264)       at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

"web-3" prio=5 tid=0x2814e470 nid=0xd70 in Object.wait() [0x2faaf000..0x2faafd98]       at java.lang.Object.wait(Native Method)       - waiting on  (a jrunx.scheduler.JSemaphore)       at jrunx.scheduler.JSemaphore.acquire(JSemaphore.java:74)       - locked  (a jrunx.scheduler.JSemaphore)       at jrun.servlet.network.NetworkService.accept(NetworkService.java:348)       at jrun.servlet.http.WebService.createRunnable(WebService.java:104)       at jrunx.scheduler.ThreadPool$DownstreamMetrics.createRunnable(ThreadPool.java:285)       at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:347)       at jrunx.scheduler.ThreadPool$UpstreamMetrics.createRunnable(ThreadPool.java:239)       at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"web-2" prio=5 tid=0x271954c0 nid=0x170c in Object.wait() [0x2ef6e000..0x2ef6fa18]       at java.lang.Object.wait(Native Method)       - waiting on  (a java.awt.image.PixelGrabber)       at java.awt.image.PixelGrabber.grabPixels(PixelGrabber.java:254)       - locked  (a java.awt.image.PixelGrabber)       at java.awt.image.PixelGrabber.grabPixels(PixelGrabber.java:209)       at com.lowagie.text.Image.getInstance(Unknown Source)       at com.lowagie.text.Image.getInstance(Unknown Source)       at com.lowagie.text.pdf.PdfGraphics2D.addAltText(Unknown Source)       at ice.pilots.html4.MacromediaCSSExtension.addonImageProcessing(MacromediaCSSExtension.java:186)       at ice.pilots.html4.ObjectBox.paintChunk(OEAB)       at ice.pilots.html4.InlineBox.paintChildren(OEAB)       at ice.pilots.html4.InlineBox.paintChunk(OEAB)       at ice.pilots.html4.InlineBox.paintChildren(OEAB)       at ice.pilots.html4.BlockBox.paint(OEAB)       at ice.pilots.html4.OutlinePainter.drawBox(OEAB)       at ice.pilots.html4.BlockBox.paint(OEAB)       at ice.pilots.html4.OutlinePainter.drawBox(OEAB)       at ice.pilots.html4.BlockBox.paint(OEAB)       at ice.pilots.html4.CSSLayout.paint(OEAB)       - locked  (a java.lang.Object)       at ice.pilots.html4.ThePrinter.printPage(Unknown Source)       at coldfusion.document.DocumentSection.process(DocumentSection.java:230)       at coldfusion.document.DocumentSection.print(DocumentSection.java:108)       at coldfusion.document.DocumentExporter.export(DocumentExporter.java:237)       at coldfusion.document.DocumentFrame.exportContent(DocumentFrame.java:118)       at coldfusion.document.DocumentProcessor.processContent(DocumentProcessor.java:130)       at coldfusion.document.DocumentProcessor.ProcessContent(DocumentProcessor.java:59)       at coldfusion.tagext.lang.DocumentTag.processContent(DocumentTag.java:1218)       at coldfusion.tagext.lang.DocumentTag.access$100(DocumentTag.java:84)       at coldfusion.tagext.lang.DocumentTag$3.run(DocumentTag.java:1179)       at java.security.AccessController.doPrivileged(Native Method)       at coldfusion.tagext.lang.DocumentTag.doAfterBody(DocumentTag.java:1174)       at cfcmyk2ecfm937582361.runPage(E:\CFusionMX7\wwwroot\testfolder\doctest\cmyk.cfm:3)       at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:152)       at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:349)       at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)       at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:209)       at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:51)       at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)       at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:69)       at coldfusion.filter.ClientScopePersistenceFilter
.invoke(ClientScopePersistenceFilter.java:28)       at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)       at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)       at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)       at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:115)       at coldfusion.CfmServlet.service(CfmServlet.java:107)       at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:78)       at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:91)       at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)       at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:257)       at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:541)       at jrun.servlet.http.WebService.invokeRunnable(WebService.java:172)       at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:318)       at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:426)       at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:264)       at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)...

If you anayze this thread dump, you can see that “web-2″ thread is handling request for “cmyk.cfm” and is waiting for the images to be loaded i.e (actually at Image.getInstance()). At this point, cfdocument has sent another http request for the image file that it has to render and that request is handled by another thread “web-4″ which is running and is currently checking if the image file requested exists on the file system. Isn’t it very powerful ?

You can checkout a very nice article Thread dump and stack traces written by Rajiv (my ex-colleague and mentor in Pramati) which I feel is a must read for Java developers.

Tags: , ,