Merge pull request #931 from carsonip/fix-rdkafka-setup

Use env var to explicitly specify rdkafka setup

Author: Emmett Butler

README.rst

@@ -159,8 +159,16 @@ Using the librdkafka extension
 ------------------------------
 
 PyKafka includes a C extension that makes use of librdkafka to speed up producer
-and consumer operation. PyKafka requires librdkafka v0.9.1+. Some system package managers may 
-not have up-to-date versions. To use the librdkafka extension, you need to make sure the header
+and consumer operation.
+
+To ensure the C extension is compiled, set environment variable ``RDKAFKA_INSTALL=system`` during
+``pip install`` or ``setup.py``, i.e. ``RDKAFKA_INSTALL=system pip install pykafka``.
+The setup will fail if C extension is not compiled. Oppositely, if ``RDKAFKA_INSTALL=''``, this
+explicitly specifies that the C extension should not be compiled. The current default behavior is
+to compile the extension but will not fail the setup if compilation fails.
+
+PyKafka requires librdkafka v0.9.1+. Some system package managers may not have
+up-to-date versions. To use the librdkafka extension, you need to make sure the header
 files and shared library are somewhere where python can find them, both when you build
 the extension (which is taken care of by ``setup.py develop``) and at run time.
 Typically, this means that you need to either install librdkafka in a place

setup.py

@@ -184,8 +184,19 @@ def run_setup(with_rdkafka=True):
  ]
  )
 
+
+# Use environment variable RDKAFKA_INSTALL to explicitly specify whether rdkafka c extension should be compiled
+# 'system': Compile with librdkafka installed in system
+# '' (empty string): No compile
+rdkafka_install_type = os.environ.get('RDKAFKA_INSTALL')
+
 try:
- if not is_cpython:
+ if rdkafka_install_type == '':
+ run_setup(with_rdkafka=False)
+ elif not is_cpython:
+ if rdkafka_install_type == 'system':
+ raise Exception("librdkafka is not supported under %s, but RDKAFKA_INSTALL specified"
+ % python_implementation)
  print("librdkafka is not supported under %s" % python_implementation)
  print(15 * "-")
  print("INFO: Failed to build rdkafka extension:")
@@ -195,6 +206,8 @@ def run_setup(with_rdkafka=True):
  else:
  run_setup()
 except ve_build_ext.BuildFailed as exc:
+ if rdkafka_install_type == 'system':
+ raise
  print(15 * "-")
  print("INFO: Failed to build rdkafka extension:")
  print(exc.cause)