TL;DR: a free open source software package with buried install routine that tries to AWS Cloud Formation a server cluster… would incur ~$22K/month.
In recent weeks, I’ve been reviewing dozens of open source applications. Some of the stuff I’ve found deeper in the details has been truly amazing.
The best one so far has been an open source map rendering application. The concept is pretty straightforward… it provides a client app and a server app. On the client, a user creates a set of points (coordinates), and the server renders a map for the client.
The client/server model should enable multiple users to utilize low powered mobile devices to gather data and let one server do the heavier processing.
Sounds good; I added it to my “evaluations” list.
My primary goal for these evaluation activities is not to install/run these apps, but rather to collect some apps that I can use as working examples for my own software development “continuing education” projects. I learn more by taking things apart.
After some preliminary reviews, fiddling around with a live online demo site, and using some packaged bitnami/docker type bundles… I git cloned the source code and started exploring.
As I followed a few threads of packages dependencies, I stumbled into a package of utilities and scripts… best described as “deploy an n-tiered high capacity solution.”
Notice that I found this in package dependencies and scripts.
Many app developers provide this kind of info in documentation. Not these folks. It’s deep in the installation/deployment code published in their GitHub repo.
So what happens when someone downloads and deploys the content of their GitHub repo. For most people, I’d expect the process to fail rather quickly.
However, if attempted on a host that is used to admin/deploy stuff on AWS… better hope someone configured AWS roles and account limits/restrictions.
If you’ve ever setup “AWS CLI” to make it easier to administer AWS services… and then ran the command line “aws configure” to set up your keys… well… you should be getting a feeling that a train wreck is leaving the station.
That client/server deployment process buried in the GitHub repo… it contains cloud formation scripts to deploy a “high capacity rendering cluster”.
After I reviewed the target configuration, and looked up the current AWS pricing, I estimated the services turned on would have a monthly cost of about $22K.
That isn’t the only open source repo I’ve found that hides a narly surprise, but it’s currently the top of my list.
A couple others, that take somewhat smaller bite, but nevertheless demonstrate why caution is necessary.
A package that is described as a howto guide walks the user through steps to set up a free Azure Functions demonstration… at the completion, the user account is provisioned with a persistent (always on) service that prices out as $1,803 per month. Unfortunately, the Microsoft Azure admin interface does nothing to warn the user or even ask permission. I only found that little gift by thoroughly reviewing all of the created services afterwards… and promptly deleting them. Had something external interrupted me, and I’d delayed that post config review, the unexpected bill would have been a real kick in the backside.
In another open source git repo, I found what appeared to be a promising little utility for scraping news sites and parsing the results into simple/clean text saved locally for offline reading. Turns out that streamlined reading material comes at a cost. The app’s install routine included a “pip install” of numpy, matplotlib, scipy, and tensorflow. Congratulations to anyone who installed this one… you just donated your machine to the authors machine learning network.
If you’re thinking , “Gee, Wally how many people actually make these kinda mistakes? Don’t detect them? And, don’t clean them up?”
Google search “wasted cloud spend”.
An article published Jan 3rd, 2019 puts the current cloud waste at $14.1 Billion.