Development Sandbox: Coding Patterns

Java is a habit-forming programming language. After a few years, good programmers find efficient ways to code. They’ll accept the programming language’s shortcomings. And they’ll make the best of a not-so-good situation. Pair programming and code reviews have provided a means for programmers to share ideas. They learn from each other to write better and cleaner code. It’s a social experiment of sorts. The more you are exposed to other developers and their code, the more your skills improve. It’s an evolution. Continue reading “Development Sandbox: Coding Patterns”

Development Sandbox: Int to String Conversion

Recently, while reviewing some legacy code I discovered a trick a developer had used to convert an integer to a string. The developer did this because the function only accepted a string as a parameter. As developers, often we look for succinct ways to code thinking that less code is faster. The interesting thing about the code is that it was the slowest of all the tests I had done. In all my efforts, I found five different ways to code to a conversion of an integer to a string. Continue reading “Development Sandbox: Int to String Conversion”

Development Sandbox: String Concatenation

When you review and work with other people’s code you sometimes find some tricks to optimize your own code. Most of the time the tricks look impressive in their succinctness and streamlined approach. And so you’d assume that the performance behind the scenes would be mind-blowing. So I decided to have a look at string concatenation. I have seen a number of ways to concatenate strings. Usually, the ones that do everything within scope will do it properly. And there are concatenations done other ways for appropriate reasons. But I thought I’d still have a look at the many ways one can concatenate a string.

Continue reading “Development Sandbox: String Concatenation”

Development Sandbox: Java code inlining

Once in a while, it’s fun to take out the Java development tools and experiment. So I did just that. I went online and found a bytecode editor that does a quick job of parsing Java class files. I wanted to see what my code looked like in bytecode form. I wanted to test if a complicated if condition would get inlined. You know what I’m talking about. You’ve written that complicated condition fairly often. You need to validate that the variable you’re testing against is not null and is accessible.

Below is an example of such a condition:

arrayOfValues != null && arrayOfValues.length > 0 && arrayOfValues[0].equals(Integer.toString(anotherValue))

When a condition gets ugly I will often extract it into its own method to make the code easier to read. It also provides the opportunity to do some intense unit testing.

So here is the sample code I wrote that I would be testing:

public class Sample {
    // The number of iterations for the for loop
    static int iterations = 1000000;
    // How often do we print the iteration
    static int step = 100000;

    // The variable in our test
    static String[] arrayOfValues = {"1353"};

    public static void main(String... args) {
        // This is the big loop
        for (int i = 0; i < iterations; i++) {

            // Notify us of where we are in the loop
            if (i % step == 0) {
                System.out.println("Iteration: " + i);
            }

            // Notify us when we've match with the iteration
            if (isAMatch(i)) {
                System.out.println("Found: " + i);
            }
        }
    }

    // This is our test condition
    private static boolean isAMatch(int i) {
        return arrayOfValues != null && arrayOfValues.length > 0 && arrayOfValues[0].equals(Integer.toString(i));
    }
}

The code isn’t complicated so I won’t explain what it’s doing. But note the extracted method isAMatch() it’s an example of what most developers would code.

By default, the JIT will inline methods that are 34 bytes or shorter. So I used the Java Bytecode Editor to review the class file and see what’s what. According to the editor the function isAMatch() is just about 34 bytes long; just within the maximum limit. I’ll be honest, I thought Java would be able to fit more within 34 bytes. In a later post, I’ll experiment with different ways of writing the condition to see if I can get it shorter. But for now, it seems that the function is a good candidate for inlining.

javabytecodeeditor-1

On to testing

I compile the code using the Java 8 JDK with the following statement. Nothing extravagant.
javac Sample.java
The fun starts when executing the application, that’s when the JIT does all of its optimization magic. Below is the command I used to execute the class file.
java -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining Sample
This is what those options do:
  • UnlockDiagnosticVMOptions: This option it turns on the diagnostic tracking. Oddly enough, it is not listed on Oracle’s website even though it is required for PrintInlining to work. But the JIT will display an error if it’s not provided.
  • PrintInlining: This option will display a report on all the decisions the JIT makes in regards to inlining methods. It is impressive to see in action.
When you execute the class this is the output that you’ll get.

 

</pre>
@ 1 java.lang.Object::<init> (1 bytes)
@ 13 java/lang/StringIndexOutOfBoundsException::<init> (not loaded) not inlineable
@ 30 java/lang/StringIndexOutOfBoundsException::<init> (not loaded) not inlineable
@ 65 java/lang/StringIndexOutOfBoundsException::<init> (not loaded) not inlineable
@ 75 java.util.Arrays::copyOfRange (63 bytes) callee is too large
@ 16 java.lang.StringBuilder::<init> (7 bytes)
@ 3 java.lang.AbstractStringBuilder::<init> (12 bytes)
@ 1 java.lang.Object::<init> (1 bytes)
@ 20 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (62 bytes) callee is too large
@ 25 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (50 bytes) callee is too large
@ 29 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (62 bytes) callee is too large
@ 32 java.lang.StringBuilder::toString (17 bytes)
@ 13 java.lang.String::<init> (82 bytes) callee is too large
@ 35 java.lang.IllegalArgumentException::<init> (6 bytes) don't inline Throwable constructors
@ 54 java.lang.Math::min (11 bytes)
@ 57 java.lang.System::arraycopy (0 bytes) intrinsic
@ 66 java.lang.String::indexOfSupplementary (71 bytes) callee is too large
@ 18 java/lang/StringIndexOutOfBoundsException::<init> (not loaded) not inlineable
@ 4 java.lang.CharacterDataLatin1::getProperties (11 bytes)
@ 1 java.lang.Character::toUpperCase (9 bytes)
@ 1 java.lang.CharacterData::of (120 bytes) callee is too large
@ 5 java.lang.CharacterData::toUpperCase (0 bytes) no static binding
@ 1 java.lang.CharacterData::of (120 bytes) callee is too large
@ 5 java.lang.CharacterData::toUpperCase (0 bytes) no static binding
@ 1 java.lang.CharacterData::of (120 bytes) callee is too large
@ 5 java.lang.CharacterData::toLowerCase (0 bytes) no static binding
@ 4 java.lang.CharacterDataLatin1::getProperties (11 bytes)
@ 3 java.lang.String::indexOf (70 bytes) callee is too large
@ 17 java.lang.AbstractStringBuilder::newCapacity (39 bytes) callee is too large
@ 20 java.util.Arrays::copyOf (19 bytes)
@ 11 java.lang.Math::min (11 bytes)
@ 14 java.lang.System::arraycopy (0 bytes) intrinsic
@ 7 java.lang.AbstractStringBuilder::ensureCapacityInternal (27 bytes)
@ 17 java.lang.AbstractStringBuilder::newCapacity (39 bytes) callee is too large
@ 20 java.util.Arrays::copyOf (19 bytes)
@ 11 java.lang.Math::min (11 bytes)
@ 14 java.lang.System::arraycopy (0 bytes) intrinsic
@ 2 java.lang.AbstractStringBuilder::append (29 bytes)
@ 7 java.lang.AbstractStringBuilder::ensureCapacityInternal (27 bytes)
@ 17 java.lang.AbstractStringBuilder::newCapacity (39 bytes) callee is too large
@ 20 java.util.Arrays::copyOf (19 bytes)
@ 11 java.lang.Math::min (11 bytes)
@ 14 java.lang.System::arraycopy (0 bytes) intrinsic
@ 7 java.lang.AbstractStringBuilder::append (29 bytes)
@ 7 java.lang.AbstractStringBuilder::ensureCapacityInternal (27 bytes)
@ 17 java.lang.AbstractStringBuilder::newCapacity (39 bytes) callee is too large
@ 20 java.util.Arrays::copyOf (19 bytes)
@ 11 java.lang.Math::min (11 bytes)
@ 14 java.lang.System::arraycopy (0 bytes) intrinsic
@ 5 java.lang.AbstractStringBuilder::appendNull (56 bytes) callee is too large
@ 10 java.lang.String::length (6 bytes)
@ 21 java.lang.AbstractStringBuilder::ensureCapacityInternal (27 bytes)
@ 17 java.lang.AbstractStringBuilder::newCapacity (39 bytes) callee is too large
@ 20 java.util.Arrays::copyOf (19 bytes)
@ 11 java.lang.Math::min (11 bytes)
@ 14 java.lang.System::arraycopy (0 bytes) intrinsic
@ 35 java.lang.String::getChars (62 bytes) callee is too large
@ 2 java.lang.AbstractStringBuilder::append (50 bytes) callee is too large
Iteration: 0
@ 1 java.lang.Object::<init> (1 bytes)
@ 19 Found: 1353 java.lang.Integer::toString (48 bytes) callee is too large

@ 22 java.lang.String::equals (81 bytes) callee is too large
@ 15 java.lang.Integer::stringSize (21 bytes)
@ 24 java.lang.Integer::stringSize (21 bytes)
@ 35 java.lang.Integer::getChars (131 bytes) callee is too large
@ 44 java.lang.String::<init> (10 bytes)
@ 1 java.lang.Object::<init> (1 bytes)
@ 24 java.lang.Integer::stringSize (21 bytes) inline (hot)
@ 35 java.lang.Integer::getChars (131 bytes) inline (hot)
@ 44 java.lang.String::<init> (10 bytes) inline (hot)
@ 1 java.lang.Object::<init> (1 bytes) inline (hot)
@ 19 java.lang.Integer::toString (48 bytes) inline (hot)
@ 24 java.lang.Integer::stringSize (21 bytes) inline (hot)
@ 35 java.lang.Integer::getChars (131 bytes) inline (hot)
@ 44 java.lang.String::<init> (10 bytes) inline (hot)
@ 1 java.lang.Object::<init> (1 bytes) inline (hot)
@ 22 java.lang.String::equals (81 bytes) (intrinsic)
@ 24 java.lang.StringBuilder::<init> (7 bytes)
@ 3 java.lang.AbstractStringBuilder::<init> (12 bytes)
@ 1 java.lang.Object::<init> (1 bytes)
@ 29 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (50 bytes) callee is too large
@ 33 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (62 bytes) callee is too large
@ 36 java.lang.StringBuilder::toString (17 bytes)
@ 13 java.lang.String::<init> (82 bytes) callee is too large
!m @ 39 java.io.PrintStream::println (24 bytes)
@ 6 java.io.PrintStream::print (13 bytes)
!m @ 9 java.io.PrintStream::write (83 bytes) callee is too large
!m @ 10 java.io.PrintStream::newLine (73 bytes) callee is too large
@ 43 Sample::isAMatch (34 bytes)
@ 19 java.lang.Integer::toString (48 bytes) callee is too large
@ 22 java.lang.String::equals (81 bytes) callee is too large
@ 56 java.lang.StringBuilder::<init> (7 bytes)
@ 3 java.lang.AbstractStringBuilder::<init> (12 bytes)
@ 1 java.lang.Object::<init> (1 bytes)
@ 1 java.lang.Object::<init> (1 bytes) inline (hot)
@ 61 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (50 bytes) callee is too large
@ 65 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (62 bytes) callee is too large
@ 68 java.lang.StringBuilder::toString (17 bytes)
@ 13 java.lang.String::<init> (82 bytes) callee is too large
!m @ 71 java.io.PrintStream::println (24 bytes)
@ 6 java.io.PrintStream::print (13 bytes)
!m @ 9 java.io.PrintStream::write (83 bytes) callee is too large
!m @ 10 java.io.PrintStream::newLine (73 bytes) callee is too large
@ 15 java.lang.Integer::stringSize (21 bytes) inlining prohibited by policy
@ 24 java.lang.Integer::stringSize (21 bytes) inlining prohibited by policy
@ 35 java.lang.Integer::getChars (131 bytes) callee is too large
@ 44 java.lang.String::<init> (10 bytes)
@ 1 java.lang.Object::<init> (1 bytes)
@ 19 java.lang.Integer::toString (48 bytes) callee is too large
@ 22 java.lang.String::equals (81 bytes) callee is too large
@ 24 java.lang.StringBuilder::<init> (7 bytes)
@ 3 java.lang.AbstractStringBuilder::<init> (12 bytes)
@ 1 java.lang.Object::<init> (1 bytes)
@ 29 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (50 bytes) callee is too large
@ 33 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (62 bytes) callee is too large
@ 36 java.lang.StringBuilder::toString (17 bytes)
@ 13 java.lang.String::<init> (82 bytes) callee is too large
!m @ 39 java.io.PrintStream::println (24 bytes)
@ 6 java.io.PrintStream::print (13 bytes)
!m @ 9 java.io.PrintStream::write (83 bytes) callee is too large
!m @ 10 java.io.PrintStream::newLine (73 bytes) callee is too large
@ 43 Sample::isAMatch (34 bytes)
@ 19 java.lang.Integer::toString (48 bytes) callee is too large
@ 22 java.lang.String::equals (81 bytes) callee is too large
@ 56 java.lang.StringBuilder::<init> (7 bytes)
@ 3 java.lang.AbstractStringBuilder::<init> (12 bytes)
@ 1 java.lang.Object::<init> (1 bytes)
@ 61 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (50 bytes) callee is too large
@ 65 java.lang.StringBuilder::append (8 bytes)
@ 2 java.lang.AbstractStringBuilder::append (62 bytes) callee is too large
@ 68 java.lang.StringBuilder::toString (17 bytes)
@ 13 java.lang.String::<init> (82 bytes) callee is too large
!m @ 71 java.io.PrintStream::println (24 bytes)
@ 6 java.io.PrintStream::print (13 bytes)
!m @ 9 java.io.PrintStream::write (83 bytes) callee is too large
!m @ 10 java.io.PrintStream::newLine (73 bytes) callee is too large
@ 24 java.lang.Integer::stringSize (21 bytes) inline (hot)
@ 35 java.lang.Integer::getChars (131 bytes) inline (hot)
@ 44 java.lang.String::<init> (10 bytes) inline (hot)
@ 1 java.lang.Object::<init> (1 bytes) inline (hot)
Iteration: 100000
Iteration: 200000
@ 24 java.lang.StringBuilder::<init> (7 bytes) inline (hot)
@ 29 java.lang.StringBuilder::append (8 bytes) inline (hot)
@ 33 java.lang.StringBuilder::append (8 bytes) executed < MinInliningThreshold times
@ 36 java.lang.StringBuilder::toString (17 bytes) executed < MinInliningThreshold times
!m @ 39 java.io.PrintStream::println (24 bytes) executed < MinInliningThreshold times
@ 43 Sample::isAMatch (34 bytes) inline (hot)
@ 19 java.lang.Integer::toString (48 bytes) inline (hot)
@ 24 java.lang.Integer::stringSize (21 bytes) inline (hot)
@ 35 java.lang.Integer::getChars (131 bytes) inline (hot)
@ 44 java.lang.String::<init> (10 bytes) inline (hot)
@ 1 java.lang.Object::<init> (1 bytes) inline (hot)
@ 22 java.lang.String::equals (81 bytes) (intrinsic)
Iteration: 300000
Iteration: 400000
@ 19 java.lang.Integer::toString (48 bytes) inline (hot)
@ 24 java.lang.Integer::stringSize (21 bytes) inline (hot)
@ 35 java.lang.Integer::getChars (131 bytes) inline (hot)
@ 44 java.lang.String::<init> (10 bytes) inline (hot)
@ 1 java.lang.Object::<init> (1 bytes) inline (hot)
@ 22 java.lang.String::equals (81 bytes) (intrinsic)
Iteration: 500000
Iteration: 600000
Iteration: 700000
Iteration: 800000
Iteration: 900000

What I noticed is that the method isAMatch() is not inlined until it has gone through at least 200,000 iterations as noted by the label inline (hot) next to the method isAMatch() on line 160. To me that says three things.
First, the JIT thinks the method executes fast enough that it didn’t warrant inlining the first 200,000 iterations.
Second, a lot of smaller methods are inlined before the code reaches 100,000 iterations.
Third, I’m guessing that smaller methods are inlined first simply due to overhead in calling them. The overall performance of the application is what’s most important. So JIT thought that my method could wait.

It’s important to note that there are JIT options to control the maximum byte size of methods to inline. What I would like to investigate is how inlining plays out with regards to natively compiled code. And when does the JIT think it necessary to compile bytecode to native code. All fun stuff.  So stay tuned for more on this.

Using scripts to improve your workflow

Development these days is very complicated. You have to contend with many types of technologies. Source code management. Maven or Gradle build scripts. Testing of all sorts. Deployment to various destinations. The permutations are endless. CI/CD (continuous integration & continuous deployment) tools like Jenkins can handle several of these challenges. But sometimes all you need is a simple script to do the job.

For example, on I’ve written different versions of the same script. They all sit on my desktop and I just click on the one that I need when I need it. I may have one that executes Maven to performs unit tests on the whole suite. Or one that only executes tests on a subset. I have also written a script that builds and deploy to a JBoss server without doing the unit tests first. This can save a lot of time when you’ve already executed your tests, fixed code, test, fix code in endless iterations. You don’t want to run those tests all over again. So script can be handy for all kinds of tasks.

Apache Spark

I’m currently learning Apache Spark running in a VirtualBox VM on a Windows host. Needless to say, scripting here is indispensable. I’ve written a script to start the VM in headless mode with a static IP address. Once the VM is up the script launches an SSH session for me.

What is really priceless is the example below of a BASH script that I put together to deploy an application into the Spark Server. This script works well because I just need to double-click on the script and it runs.  What the script does is nothing miraculous except that it simplifies a convoluted process that gets performed often.

As this example. all my scripts are written in BASH and execute in Git BASH from the Windows desktop. The script below will first build the application with code that currently checked-out. Then uploads the resulting JAR file to the VM using the SCP command. Once on the server, the JAR is submitted to Spark using the SSH command to perform a remote CLI call. Finally, once that process is done, another SCP call is made to retrieve the resulting CSV file.

# Change to the project directory
cd /c/dev/projects/apache-spark

# Build the application into a JAR file
mvn clean install

# Upload the file to the VM
scp -i /c/Users/Don/.ssh/id_rsa -P 22 /c/dev/projects/apache-spark/target/jar/SparkApp.jar spark@192.168.0.50:/home/spark/deployment/SparkApp.jar

# Submit the JAR to the Spark server
ssh -i /c/Users/Don/.ssh/id_rsa -p 22 spark@192.168.0.50 '/home/spark/spark/bin/spark-submit -v --class com.sparkinaction.App --master local /home/spark/deployment/SparkApp.jar'

# Retrieve result and copy to project folder
scp -i /c/Users/Don/.ssh/id_rsa -P 22 spark@192.168.0.50:/home/spark/result/sightings.csv /c/dev/projects/apache-spark/target/spark-result/result.csv

For those that don’t work in Linux much, there are two commands that I find indispensable.

SCP – This command copies files between computers using SSH. It’s secure and allows me to log in using an SSH key.

SSH – Everyone knows SSH, but did you know you can use it to execute commands remotely. That’s what I did above to submit the Java application to the Spark server.

Artificial Intelligence in the News – April 2017

Artificial Intelligence is certainly a strange beast. I’ve read some really interesting articles on the subject recently on how creative and aggressive they can be. Below are the more interesting ones I’ve read.

AI does Knock Knock Jokes

AI Gets Aggressive when playing games

Japanese AI Writes a Novel, Nearly Wins Literary Award

Listen to New Google AI Program Talk Like a Human and Write Music

Can artificial intelligence read a tweet’s sentiment?

Learning the Basics of Machine Learning

Educating yourself on machine learning is a challenging preposition. There is so much content online that it’s easy to get fooled into reading an article that is beyond capabilities. Unfortunately, in many cases, reading and rereading this sort of content is often the only way to crack that nut open. But recently I’ve found some great resources that should hopefully be able to help the uninitiated into getting a foothold into this popular segment of IT. Continue reading “Learning the Basics of Machine Learning”