Capture HTTPS traffic from Java applications with Fiddler

Fiddler usually works out of the box, with a few exceptions. One of those exceptions is capturing traffic from a JVM.

To capture plain HTTP traffic from a JVM, you can configure Fiddler as the proxy by setting these VM args:

-DproxySet=true
-DproxyHost=127.0.0.1
-DproxyPort=8888

[Note: Fiddler proxies at port 8888 by default]

Capturing HTTPS traffic (of course, to view it unencrypted in Fiddler), is slightly more involved. Here’s how to do that.

1. Export Fiddler’s Root Certificate

Click on Tools -> Fiddler Options… to open the Fiddler Options dialog.

Fiddler Options Menu

 

Switch to the HTTPS tab, and click on Export Root Certificate to Desktop.

Fiddler Options Dialog - HTTPS Tab

This will generate the file: FiddlerRoot.cer on your Desktop.

2. Create a JVM Keystore using this certificate

This step will require Administrator privileges (since keytool doesn’t seem to work without elevating privileges). So, open command prompt as Administrator, by right clicking on the Command Prompt icon, and clicking on Run as administrator.

Run the following command (replacing <JAVA_HOME> with absolute path to the JDK/JRE that you’re interested in capturing traffic from):

<JDK_Home>\bin\keytool.exe -import -file C:\Users\<Username>\Desktop\FiddlerRoot.cer -keystore FiddlerKeystore -alias Fiddler

This will prompt you to enter a password. Remember the password, as it’s required for the next step.

Once a password is entered, this’ll create a file called FiddlerKeyStore. Remember the path to this file, as we’ll be using it in the next step. You can, of course, move it to a more convenient location and use that path.

3. Start the JVM with Fiddler as the proxy, and the Keystore you just created as a Trust Store

Essentially, we’re asking the JVM to use Fiddler as the proxy, and to trust the keys in the Keystore we just created. Here’re the VM args to configure your Keystore as the Trust Store:

-Djavax.net.ssl.trustStore=<path\to\FiddlerKeystore>
-Djavax.net.ssl.trustStorePassword=<Keystore Password>

 

So, in effect, these are the VM args you’ll need:

-DproxySet=true
-DproxyHost=127.0.0.1
-DproxyPort=8888
-Djavax.net.ssl.trustStore=<path\to\FiddlerKeystore>
-Djavax.net.ssl.trustStorePassword=<Keystore Password>

That’s about it. Now, launch Fiddler, and launch your JVM (your Java application). Fiddler will start showing all HTTPS (and HTTP) traffic from the JVM in plaintext.

Setting up VM args in an IDE

If you’re using IntelliJ or Eclipse for development, you can set VM args in the Run Configuration dialogs. Here’s what they look like with the VM args populated:

Here’s the IntelliJ Run Configuration dialog with the VM args populated:

IntelliJ Run Configuration with VM args

 

And here’s the Eclipse Run Configuration dialog with the VM args populated:

Eclipse Run Configuration with VM args

 

[PS: This post is based on one of my SO answers: http://stackoverflow.com/questions/8549749/how-to-capture-https-with-fiddler-in-java/8588914#8588914]

Missing ldconfig and /etc/sudoers

I was trying to upgrade my Ubuntu installation to 12.10 (Quantal Quetzal), and the update manager (Muon) kept failing with error messages like:

dpkg: warning: 'ldconfig' not found in PATH or not executable.
dpkg: warning: 'start-stop-daemon' not found in PATH or not executable.
dpkg: error: 2 expected programs not found in PATH or not executable.
Note: root's PATH should usually contain /usr/local/sbin, /usr/sbin and /sbin.

which ldconfig as root returned /sbin/ldconfig, and of course, root‘s $PATH had /sbin in it, so, couldn’t think of a reason why the updates were failing. But a bit a googling led me to the sudoers file (/etc/sudoers)

Muon, and in turn apt-get, use sudo for installing stuff. And sudo starts with an empty/default ENV, if it’s either been compiled with –with-secure-path, or if env_reset has been set in the sudoers file. In my case, env_reset was set in the sudoers file, so, sudo‘s ENV didn’t have /sbin in it. In case of env_reset, you should provide a secure_path, which’s the $PATH that any sudoed process would use. So, after the fix, my /etc/sudoers looks like:

Defaults env_reset
Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

In case you’re seeing the error, but don’t have env_reset set in your sudoers file, then it’s likely that your version of sudo was compiled with -with-secure-path. To see the options that your version of sudo was compiled with:

$ cat /usr/share/doc/sudo/OPTIONS

Bypassing SSL certificate checks during gem installation

I had to re-install rvm on my MacBook because my gemsets were a bit messed up, and I thought I should start over with a clean install. I just rm -rf‘ed ~/.rvm, and then went ahead and re-installed it according to the instructions at https://rvm.io/. Installed ruby 1.9.3, and the openssl package as described here. But after that, both gem install and bundle install started failing because the SSL certificate from https://rubygems.org couldn’t be verified. Couldn’t quite figure out what was wrong. But, after a bit of googling, found out a way to skip the SSL certificate checks.

To skip the SSL certificate checks, just add this line to your .gemrc

:ssl_verify_mode: 0

This causes the gem and bundle commands to skip SSL certificate verifications when fetching them from a HTTPS source.

Of course, you can also bypass the error by using a non HTTPS URLs for your gem sources in your Gemfile (when using bundler). So, something like:

source 'https://rubygems.org'

in your Gemfile, will become:

source 'http://rubygems.org'

 

Neither of these actually fix the problem. They just avoid SSL certificate checks, or use a non SSL source. I still don’t know what went wrong during the re-install.

 

Ruby 1.9.3 segfault in OSX Lion due to OpenSSL

I just set-up my new MacBook Pro, running OSX Lion (10.7.3), for Rails development. Installed rvm, installed Ruby 1.9.3 using rvm and did a gem install rails -v 3.2.3. Everything went fine, until I tried to create a new rails app.

rails new <app_name> would create the app structure, but would then fail with a segfault in http.rb. Here’s what the stacktrace looked like:

...
run  bundle install
/Users/CodeMangler/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799: [BUG] Segmentation fault
ruby 1.9.3p125 (2012-02-16 revision 34643) [x86_64-darwin11.3.0]

— Control frame information ———————————————–
c:0038 p:—- s:0217 b:0217 l:000216 d:000216 CFUNC :connect
c:0037 p:0011 s:0214 b:0214 l:001570 d:000213 BLOCK /Users/CodeMangler/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799

So, the failure was while doing a bundle install at the end of creating a new rails application, which’s new in rails (to me anyway. It’s been a while since I last looked into rails).
Googling for ruby segfault in http.rb pointed towards issues with OpenSSL. Then I found this post that describes how to fix the issue: http://www.rojotek.com/blog/2012/01/20/how-to-get-openssl-in-ruby-1-9-3-working-on-osx-10-7-fixing-the-segmentation-fault-with-ruby-openssl/

It tuns out, you’ve to install the openssl package for rvm, and then, while installing ruby, point it to use this version of OpenSSL.

To quote steps from the post:

$ rvm pkg install openssl
$ rvm remove 1.9.3 # uninstall the existing version, if you've installed one
$ rvm install 1.9.3 --with-openssl-dir=$rvm_path/usr --with-gcc=clang

Do that, and you should be all set. I tried creating a new rails app after that, and it went through just fine. Also, from googling, it appears that if it’s a segfault in http.rb, it’s most likely due to OpenSSL.