Tuesday, August 16, 2011

Java ProxySelector, useSystemProxies and new Socket() = Malformed reply from SOCKS server

When you want to easily configure your java application to use the systems proxies and you do a little research on google, you will find some code snippets that look pretty much like this (from http://stackoverflow.com/questions/376101/setting-jvm-jre-to-use-windows-proxy-automatically):

System.setProperty("java.net.useSystemProxies", "true");
System.out.println("detecting proxies");
List l = null;
try {
    l = ProxySelector.getDefault().select(new URI("http://foo/bar"));
} catch (URISyntaxException e) {
    e.printStackTrace();
}
if (l != null) {
    for (Iterator iter = l.iterator(); iter.hasNext();) {
        java.net.Proxy proxy = (java.net.Proxy) iter.next();
        System.out.println("proxy hostname : " + proxy.type());

        InetSocketAddress addr = (InetSocketAddress) proxy.address();

        if (addr == null) {
                System.out.println("No Proxy");
        } else {
                System.out.println("proxy hostname : " + addr.getHostName());
                System.setProperty("http.proxyHost", addr.getHostName());
                System.out.println("proxy port : " + addr.getPort());
                System.setProperty("http.proxyPort", Integer.toString(addr.getPort()));
        }
    }
}

This works fine, as long as you only use it to determine the System proxy for URLConnections.

However simply opening a new Socket with a command like this

Socket tunnel = new Socket("192.168.1.101", 8080);

fails with an exception if the above code has been executed before:

java.net.SocketException: Malformed reply from SOCKS server
at java.net.SocksSocketImpl.readSocksReply(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)

In my case, I was using apache axis (axis1) which I had to use because my webservice only supported the rpc/literal SOAP encoding. After looking into this issue for a while, I found the solution in the piece of code from above. It appears java.net.Socket always uses a proxy when a default ProxySelector is set.

I am unsure of why this happens, but what fixed the issue for me was to set the default proxy selector to null after I retrieved the system proxy with the above code:

ProxySelector.setDefault(null);