Raise exception when Maven artifacts fail download in setup.py

[jcarey03]

Problem
During pip install if a Maven artifact fails to download, the failure is silent (i.e., only a print message occurs). This results in subtle undesirable behavior in KCL workers where a subset of jars exists on the classpath.

Solution
Replace the print message with a RuntimeError for better awareness of failed downloads.

[jcarey03]

Tagging @pfifer for visibility.

[pfifer]

It appears the source branch isn't up to date with master. Could you pull master into your branch re-push the PR?

[jcarey03]

@pfifer, sorry about that. We have a fork and it was out of date. Master is now merged. Thanks!

[jcarey03]

@pfifer, would you mind taking another look? Thanks!

[pfifer]

Does this cause pip to stop the installation when the download fails?

[jcarey03]

So, a pip install of the dependency directly does in fact fail, which is the desired behavior. This is what I was testing originally, but I noticed upon closer inspection, a pip install with a requirements file does not fail. I'm looking into the difference between the following:

// pip install direct dependency fails correctly
pip -v install git+https://github.com/lyft/amazon-kinesis-client-python@exc_on_failed_jar_download_testing#egg=amazon-kinesis-client-python

// pip requirements file does not fail
pip -v install -r ./requirements.txt

This is probably something obvious, but I'm new Python :)

[pfifer]

Thanks just need a license confirmation before I can start merging the change.

Please confirm that we can use, modify, copy, and redistribute this contribution.

Thanks.

[jcarey03]

Thanks. I'll confirm.

Before merging, I want to understand why this doesn't fail when specifying a requirements file. I'm looking into it this week.

[pfifer]

I did some testing with requirements.txt and virtualenv. It did throw an error during that testing, it's possible that you didn't actually run your version of the install during the requirements.txt test. My normal testing for installation is something like:

1. Install the current version, and make sure all the dependencies are installed:
   pip install amazon_kclpy
2. Remove the current version, this leaves all the dependencies installed:
   pip uninstall amazon_kclpy
3. Make sure the new version is built:
   python setup.py sdist
4. Attempt to install the new version:
   pip install amazon_kclpy --no-index --find-links <path to the repo>/amazon-kinesis-client-python/dist

Step 4 can work for requirements.txt as well
pip install -r requirements.txt --no-index --find-links <path to the repo>/amazon-kinesis-client-python/dist

I've had issues with the wheel cache messing this up on Python 3, and if you don't see the process running try adding --no-cache-dir after pip

[jcarey03]

Thanks for the detailed steps! I was able to reproduce the failure when providing a requirements.txt file.

I have reached out to our Legal team to look into allowing you to use, modify, copy, and redistribute this contribution. I will provide updates when I have them.

[jcarey03]

@pfifer I just received confirmation on my end that you may use, modify, copy, and redistribute this contribution. Apologies for the delay while we got the legal stuff in order.

[jcarey03]

@pfifer, PTAL when you can. Once this merges, when would be the next release of the library?