You can build AI
Java Developer? Run Image Classification on AWS Lambda using DJL
We are combining Java, the most popular programing language, with deep learning, the most exciting technology today, on AWS Lambda, the most cost-efficient environment of serverless in the cloud. Moreover, it will take less time than preparing a cup of Java, and will cost less than the recycled paper cup to use for it.
Why Java?
The religious war of “what is the best programming language” will probably never be over. Usually, the best language is the one that you know best. It is hard to write poetry or even to read poetry in a language that you just learned. The limitations of using less familiar languages are valid for an individual software developer and even more for a software development team of a large enterprise.
I’m working with many companies that had embarked on the journey to data, analytics, and AI transformations. Beyond the fancy buzzwords lies a tremendous shift in the technological world from the age of digital transformation (less paperwork and more digital records) to the age of advanced analytics (evidence-based and data-driven decisions). It is not enough today to install a database and develop an application to read data from and to write data to it. Now the IT and software teams are expected to deliver decision support systems based on big data, machine learning, and other new technologies. Many of these organizations use Java as the primary programming language thanks to its 20 years of maturity, broad community, development tools, and libraries.
When AWS launched Lambda in 2015, it supported only JavaScript. I was amazed to hear from almost every big company I met, trying to explain the great innovation of Lambda service to potential customers, that “if it is not supporting Java, it can only be used for small side projects in our company. Business applications are developed in Java here”. Even today when I teach deep-learning classes for developers, I mostly see developers with ten years of Java experience and three months of Python, that they just started to pick up. Therefore, the ability to begin using deep learning in Java is useful for individuals and enterprises alike to allow them to focus on the AI aspects without the complexity of learning to use a new language for production/business applications.
Why AWS Lambda?
AWS Lambda is one of the most innovative services of the AWS cloud. It is the “cloudiest” service, where you don’t need to care not just about the hardware you are using (virtual environment), the operating systems (container), the webserver, and the router leading to it, but about any other aspect of computing that is needed to execute the function. It is also the “cloudiest” service in terms of cost, where you pay for milliseconds of execution, and nothing for idle time. No wonder that the term “serverless” is associated with Lambda, more than any other service.
Lambda is not the best computing service for every use case, but for many “long tail” functions, it is better than any other. For our use case, of getting started with deep learning, it is making it easy to have operational AI in enterprise environments that are still experimenting with this new technology.
Why Deep Learning?
There is a lot of hype around machine learning, deep learning, and artificial intelligence these days. These are not entirely new technologies; however, in the last few years, they made significant leaps in terms of accuracy, performance, frameworks support, and tools maturity in many domains such as computer vision, natural language understanding, time-series, and others. You can now integrate inputs from cameras (video, satellites, or mobile phones), text and speech messages and documents, and many other unstructured data, as data sources for advanced analytics.
Deep learning is still young, but it is already providing “better-than-human” performance on many tasks that are important for businesses, such as image classification and natural language understanding. It is also enjoying a quality that we never had with machines and systems built in the past — the “Flywheel Effect.” It is hard to get the system to work with lots of data and lots of experiments, but once it is working, it can just get better by having more and new data. Most of the systems that we built in the past started to deteriorate the minute they left the factory. Deep learning is the first technology that is potentially getting better the more you use it.
Why Deep Learning on Lambda using Java?
The journey deeper into deep learning and its business application is lengthy and requires a lot of education and many trials both from the organization and the individuals. Nevertheless, deploying a deep learning model in 5 minutes that costs zero cents is an excellent first step in the right direction.
You can take a model that was developed and trained by Amazon, Google, and others (as we will do in this example), or take one of the models that were created by your data science team, and you couldn’t deploy it to production quickly before. In most companies, the development of deep learning and AI models is done by separate research teams, using different tools and programming languages (mainly Python). Often, these models are staying in the research labs and not deployed to production environments, partly because of the reality of “our production stack is Java.” As you can see, there are many reasons why not many companies were able to benefit from AI so far, even if they already hired data scientists.
There are also limitations to Lambda that are preventing its usage in AI use cases. Lambda is not suitable for training AI models as they require long running times and often specific hardware such as GPU that is not supported in Lamba. During the training phase, the model is processing a large amount of data (millions of images, for example), as a batch process. Even for the inference part of AI (classifying a single image, for example), Lambda limits the maximum size of a package (the code with all its dependencies and libraries). This limit is preventing from running the popular Python libraries and frameworks such as TensorFlow or PyTorch. However, with Deep Java Library — DJL, you can create a small package that can fit into the limitation of Lambda. Therefore, it is an excellent option to benefit from the simplicity and Lambda and the power of deep learning models.
Enough talking, let’s build
Installing Lambda Example
The example of running one of the best image classification model (Resnet-50) was developed by the team in AWS that is building the Deep Java Library (DJL) and is available in the GitHub repository.
git clone https://github.com/aws-samples/djl-demo.git
cd djl-demo/aws/lambda-model-serving/
./gradlew deployTriggering the Lambda Function
Here, I’ll use the AWS CLI to call the Lambda function that we deployed; however, you can use any other SDK from the web, mobile or other interfaces as needed.
> aws lambda invoke --function-name DJL-Lambda --payload '{“inputImageUrl”:”https://djl-ai.s3.amazonaws.com/resources/images/kitten.jpg"}' build/output.json{
"StatusCode": 200,
"ExecutedVersion": "$LATEST"
}
Let’s see what do we have as our response in the JSON file:
> cat build/output.json[
{
"className": "n02123045 tabby, tabby cat",
"probability": 0.48384541273117065
},
{
"className": "n02123159 tiger cat",
"probability": 0.20599405467510223
},
...
]
The Resnet-50 model can classify a thousand classes (see the full list here: https://gist.github.com/yrevar/942d3a0ac09ec9e5eb3a, for example), and most likely, some of these classes are also relevant directly to your business. If you have a fashion web site, you can classify to: ‘jersey, T-shirt, tee-shirt,’ ‘jean, blue jean, denim,’ and even ‘kimono.’ The same for grocery stores, sports goods, car parts, dog breeds, etc.
Lambda Performance Monitoring
Let’s go to the AWS Lambda console and check the performance of our image classification Lambda function:
We can see the first “cold-start” call that took more than 15 seconds, and then once the container is loaded, the next invocations took around 600 milliseconds. There are a few options to keep the Lambda function “warm”; however, in many cases, it is hardly noticed.
We can see the longer “cold-start” call at the bottom of the list of the recent invocations, and the actual billed duration in ms for the majority of the calls. Please note that I modified the memory settings of the Lambda function from the original 2GB to 1GB, saving an additional 50% of the costs. Yes, this 50% is only 14 cents for 10,000 invocations a month (after the free tier, which makes it completely free), but it is still 50% of cost-saving.
What’s Next?
Hopefully, you followed the steps above and walked now a few steps in the right direction of modernizing your career and your organization’s technical stack. You can prove to your managers and customers that AI technology is not too far for them, and to yourself that you can build AI systems. There is a good chance that in six months, you will get the opportunity to lead the AI team in your company. I already saw multiple times how these first and early steps put people ahead in new technologies.
Next, you should continue to review the Git repository of DJL. From there, you can dive into the theory and other applications of deep learning with Java or Python. The easiest way is to use Binder to access all the notebooks using the MyBinder site for DJL. If you want to have more control over your environment, the following steps will make it easier to get a cloud environment to run the various notebooks to learn the basic and advanced topics of deep learning and AI.
Installing Java 11
We will install the open-source version of Java, developed mainly by Amazon, Corretto, and we will install it on a cloud instance running Amazon Linux. We can use this environment to run various notebooks of deep learning and Java from the “Dive into Deep Learning” book and course in Java or Python.
sudo yum install java-11-amazon-corretto-headlessInstalling Java Kernel for Jupyter
Jupyter notebooks are a popular tool for data scientists with its interactivity and combination of code, execution outputs, variable values, and markdown. Jupyter has good support for many languages, including Java.
I published before the setup that we are using to combine the power of Jupyter notebooks, cloud environments, and powerful IDE (such as Visual Studio Code). Here is a link to the setup:
To add support to the Java kernel you can use the following GitHub repository.
git clone https://github.com/frankfliu/IJava.git
cd IJava/
./gradlew installKernel
java — list-modules | grep “jdk.jshell”Summary and a look ahead
In this post, we explored the reasons to take a specific technical path on yours and your organization’s journey to benefit from AI technology. This path is designed for a fast, secure, and cost-effective travel for Java developers and shops. But the journey is not ending here. In the following posts, we will explore more benefits of DJL, such as Android support to add AI to your mobile applications, federated training to improve on the privacy of your users, and wrapping multiple deep learning frameworks such as TensorFlow and PyTorch, to allow you to benefit from almost all AI research in your company and everywhere.
