The first solution is to scale up existing infrastructure – the servers company bought and maintain, then look for new “boxes” to mount in old cases. But DC capacity will end somewhere (and investment budget also). At this
The business would like to move to the cloud, in the hope that the provider can provide a bigger or (cloud sales love this) infinite capacity. Of
The bad thing (and good from
A great example of
Batch processing began with mainframe computers. Today, it still plays a central role in business, engineering, science, and other areas that require running lots of automated tasks – not only processing bills and payroll, but also calculating portfolio risk, designing new products, rendering animated films, testing software, searching for energy, predicting the weather, and finding new cures for disease. There is a tool created for scenarios like this – Microsoft Azure Batch. With Azure Batch, that power is available to you when you need it, without any capital investment.
Azure Batch runs the applications that you use on Your workstations and clusters. It is an easy way to Cloud-enable your “
1: Client application/script that interacts with the Batch and Storage services to execute a parallel workload on compute nodes (virtual machines). It runs on your local workstation or on “master node” you configure.
2: The program that runs on compute nodes in Azure to perform the actual work. In the sample below, TaskApplication.exe parses the text in a file downloaded from Azure Storage (the input file). Then it produces a text file (the output file) that contains a list of the top three words that appear in the input file. After it creates the output file, TaskApplication uploads the file to Azure Storage. This makes it available to the client application for download. TaskApplication runs in parallel on multiple compute nodes in the Batch service.
With automatic scaling, you can have the Batch service dynamically adjust the number of compute nodes in a pool according to the current workload and resource usage of your
As an example, we can use 1000 H16r (compute-optimized) Azure VM instances with RDMA, 16 Xeon E5-2667 v3 Haswell 3.2 GHz (3.6 GHz with turbo) cores, 112GB of DDR4 RAM and 2000GB of local storage each, to run Your Business Intelligence process once a Year. We will have 16.000 cores, 112TB of RAM and 2.000TB of storage. The cost of this computing power will be around 2.800 EUR/h. To calculate in your own DC, you need to invest hundreds of thousands of Euros – just for using it once a year.
Processing parallel workloads with Azure Batch is typically done programmatically by using one of the Batch APIs. Your client application or service can use the Batch APIs to communicate with the Batch service. With the Batch APIs, you can create and manage pools of compute nodes, either virtual machines or Cloud services. You can then schedule jobs and tasks to run on those nodes.