After we discussed the concepts for Building a Successful Modern Data Analytics Platform in the Cloud, it is time to architect it. This post will review the reference architectures for our scalable, flexible and robust design in both Amazon Web Services (AWS) and Microsoft Azure.

Introduction

I worked in the last 20 years with hundreds of companies to harness the ever-evolving technology tools to bring to life business ideas. Some companies were startups born to the cloud age (Waze, Viber, and JFrog, for example). Some were tech giants (Amazon and Intuit, For instance) with many technical people and software developers. Yet…


“YOU CAN BUILD AI”

I worked with dozens of companies migrating their legacy data warehouses or analytical databases to the cloud. I saw the difficulty to let go of the monolithic thinking and design and to benefit from the modern cloud architecture fully. In this article, I’ll share my pattern for a scalable, flexible, and cost-effective data analytics platform in the AWS cloud, which was successfully implemented in these companies.

TL;DR, design the data platform with three layers, L1 with raw files data, L2 with optimized files data, and L3 with cache in mind. Ingest the data as it comes into L1, and transform…


Every change introduces a high risk to an enterprise company that must be assessed before its implementation. The increasing cases of cyber-attacks by malicious hackers for ransomware are a real threat to the less cloud-experienced traditional companies. Learning how to secure their newly created cloud environment is a must first step to plan and implement. This chapter will discuss how to improve the security of the organization’s data against cyber-attacks while adding new powerful business analytical capabilities.

Introduction

In the previous chapters, we described the reasons to rethink the old data warehouse or data lake model for data analytic in a…


Image by Susan Cipriano from Pixabay

Weather is affecting many of our actions, and therefore it is a common idea to add weather data into the features of machine learning models. When will farmers buy irrigation products, construction projects order underground pipes, retailers demand more hydration products, people take a taxi ride, or guests cancel their restaurant reservations?

We, at Aiola, are helping companies solve such questions.

In this article, you will learn how to:

  • Build global weather database using Amazon Athena
  • Query the database to get different values of weather time-series for any specific location on earth
  • Add the weather time series to a forecasting…

The concepts of Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL) are essential to understand for every business executive and manager. The disrupting and evolving AI technology is changing every business today and many more in the near future. Here is a brief introduction to the essences you must know to better harness its power for your business and direct its usage within your organization.

The post is based on part of “Data and AI Ideation Workshop”, we (Aiola) deliver for many traditional companies to bootstrap significant AI transformations and improve business thinking about integrating AI.

AI is not new

Illustration of AI, ML, and DL evolution from https://developer.nvidia.com/deep-learning

Since the…


You can build AI

We are combining Java, the most popular programing language, with deep learning, the most exciting technology today, on AWS Lambda, the most cost-efficient environment of serverless in the cloud. Moreover, it will take less time than preparing a cup of Java, and will cost less than the recycled paper cup to use for it.

Photo by Marvin Meyer on Unsplash

Why Java?

The religious war of “what is the best programming language” will probably never be over. Usually, the best language is the one that you know best. It is hard to write poetry or even to read poetry in a language that you just learned. …


Connecting the best of both worlds, feature rich local IDE as Visual Studio Code and powerful cloud-based compute and storage instance, is the most productive way to develop machine learning and data analytics models and systems.

This article was developed by Dr. Yaniv Saar

Introduction

The productivity of your developers, analysts, and scientists is one of the most valuable resources that is hard to optimize. You need to joggle many balls in the air: data security, data availability, compute resources, libraries and dependencies installation, code versioning, development environment (IDE), deployment to production, and many more. …


Translating data analytics from the familiar interface of Microsoft Excel to the modern and scalable interface of Jupyter notetbooks. Here is a link to the evolving guide for Excel experts learning to expand to modern analysis environment and tools.

Why Excel?

Microsoft Excel is the most popular data analytics tool in the world for many years. Excel has many great features that are making it friendly and powerful for almost every data manipulations and analysis needed in personal and enterprise context. I still remember my amazement when I first saw Lotus 1–2–3 on a personal computer. The idea of a spreadsheet was…


My job is to get people to use data more effectively to make better decisions. It is not a simple task, especially since most of the people I work with are from Venus (read my previous story about “Data Scientists are from Mars, and Business People are from Venus”). They use data, and they understand data, but they don’t always utilize data that is not in-line with what they believe about the world.

It is especially noticeable these days when the world is very different than what we used to see before, with the spread of the Coronavirus Pandemic and…


Working from home (from https://www.lifesize.com)

The bigger problem we face with the Coronavirus economical impact

The global number of people who lost their jobs in the last month is staggering. Stories of people buying guns to defend themselves from the chaos of millions of people hungry and desperate are accumulating at a frightening rate. Some believe that more people will die from hunger or related crimes than from the actual virus in the coming months. Governments across the world are struggling to find funds to support massive unemployment. Is there no way out?

How can the world benefit from having so many people at home?

Some people can work from home as part of their job as knowledge workers, and many products such as Zoom, Slack, or…

ML-Guy

Guy Ernest is the co-founder and CTO of @Aiola, an AI company serving large enterprise AI transformation in the cloud. Guy is an ML-Hero of AWS.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store