allow defining negative conditional factors with a leading ! (tox-dev#694)

* allow defining negative conditional factors with a leading ! * add negated factor conditions changelog entry * touch up the `package_formats` idea feature draft docs - moved `package_formats` related config settings together in the example - simplified the envlist declaration - added missing expected output line * touch up factor condition config to env matching docs * add negated factor conditions docs * add Jurko to the CONTRIBUTORS file

Author: jurko-gospodnetic

CONTRIBUTORS

@@ -27,6 +27,7 @@ Jannis Leidel
 Johannes Christ
 Josh Smeaton
 Julian Krause
+Jurko Gospodnetić
 Krisztian Fekete
 Laszlo Vasko
 Lukasz Balcerzak

changelog/292.feature.rst

@@ -0,0 +1 @@
+add support for negated factor conditions, e.g. ``!dev: production_log`` - by @jurko-gospodnetic

doc/config.rst

@@ -636,34 +636,42 @@ special case for a combination of factors. Here is how you do it::
 
  [testenv]
  deps =
- py34-mysql: PyMySQL ; use if both py34 and mysql are in an env name
- py27,py36: urllib3 ; use if any of py36 or py27 are in an env name
- py{27,36}-sqlite: mock ; mocking sqlite in python 2.x
-
-Take a look at first ``deps`` line. It shows how you can special case something
-for a combination of factors, you just join combining factors with a hyphen.
-This particular line states that ``PyMySQL`` will be loaded for python 3.3,
-mysql environments, e.g. ``py34-django15-mysql`` and ``py34-django16-mysql``.
-
-The second line shows how you use same line for several factors - by listing
-them delimited by commas. It's possible to list not only simple factors, but
-also their combinations like ``py27-sqlite,py36-sqlite``.
-
-Finally, factor expressions are expanded the same way as envlist, so last
-example could be rewritten as ``py{27,36}-sqlite``.
+ py34-mysql: PyMySQL ; use if both py34 and mysql are in the env name
+ py27,py36: urllib3 ; use if either py36 or py27 are in the env name
+ py{27,36}-sqlite: mock ; mocking sqlite in python 2.x & 3.6
+ !py34-sqlite: mock ; mocking sqlite, except in python 3.4
+ sqlite-!py34: mock ; (same as the line above)
+
+Take a look at the first ``deps`` line. It shows how you can special case
+something for a combination of factors, by just hyphenating the combining
+factors together. This particular line states that ``PyMySQL`` will be loaded
+for python 3.3, mysql environments, e.g. ``py34-django15-mysql`` and
+``py34-django16-mysql``.
+
+The second line shows how you use the same setting for several factors - by
+listing them delimited by commas. It's possible to list not only simple factors,
+but also their combinations like ``py27-sqlite,py36-sqlite``.
+
+The remaining lines all have the same effect and use conditions equivalent to
+``py27-sqlite,py36-sqlite``. They have all been added only to help demonstrate
+the following:
+
+- how factor expressions get expanded the same way as in envlist
+- how to use negated factor conditions by prefixing negated factors with ``!``
+- that the order in which factors are hyphenated together does not matter
 
 .. note::
 
  Factors don't do substring matching against env name, instead every
- hyphenated expression is split by ``-`` and if ALL the factors in an
- expression are also factors of an env then that condition is considered
- hold.
+ hyphenated expression is split by ``-`` and if ALL of its non-negated
+ factors and NONE of its negated ones are also factors of an env then that
+ condition is considered to hold for that env.
 
- For example, environment ``py36-mysql``:
+ For example, environment ``py36-mysql-!dev``:
 
- - could be matched with expressions ``py36``, ``py36-mysql``,
+ - would be matched by expressions ``py36``, ``py36-mysql`` or
  ``mysql-py36``,
- - but not with ``py2`` or ``py36-sql``.
+ - but not ``py2``, ``py36-sql`` or ``py36-mysql-dev``.
 
 
 Other Rules and notes

doc/drafts/extend-envs-and-packagebuilds.md

@@ -18,7 +18,7 @@ Goal: drive building of packages and the environments needed to test them, exerc
 
 It should be possible to build other kinds of packages than just the standard sdist and it should also be possible to create different kinds of builds that can be used from different environments. To make this possible there has to be some concept of factorized package definitions and a way to match these factorized builds to environments with a similar way of matching like what is in place already to generate environments. sdist would for example would match to a "sdist" factor to only be matched against virtualenvs as the default.
 
-This could then be used to hae virtualenv, conda, nixos, docker, pyenv, rpm, deb, etc. builds and tie them to concrete test environments.
+This could then be used to have virtualenv, conda, nixos, docker, pyenv, rpm, deb, etc. builds and tie them to concrete test environments.
 
 To summarize - we would need a:
 
@@ -114,19 +114,35 @@ Illustrate how to exclude a certain package format for a factor:
 ```ini
 [tox]
 plugins=conda
-envlist={py27,py35}, py27-xdist
+envlist=py27,py35,py27-xdist
 
 [testenv]
-package_formats=sdist wheel conda
 commands = py.test
+package_formats=sdist wheel conda
 exclude_package_formats= # new option which filters out packages
  py27-xdist: wheel
 ```
 
+or possibly using the negated factor condition support:
+
+```ini
+[tox]
+plugins=conda
+envlist=py27,py35,py27-xdist
+
+[testenv]
+commands = py.test
+package_formats=
+ sdist
+ !py27,!xdist: wheel
+ conda
+```
+
 Output of `tox --list`:
 
 ```
 (sdist) py27
+(wheel) py27
 (conda) py27
 (sdist) py35
 (wheel) py35

tests/test_config.py

@@ -1376,11 +1376,20 @@ def test_factors(self, newconfig):
  a: dep-a
  b: dep-b
  x: dep-x
+ !a: dep-!a
+ !b: dep-!b
+ !x: dep-!x
  """
  conf = newconfig([], inisource)
  configs = conf.envconfigs
- assert [dep.name for dep in configs['a-x'].deps] == ["dep-all", "dep-a", "dep-x"]
- assert [dep.name for dep in configs['b'].deps] == ["dep-all", "dep-b"]
+ expected = ["dep-all", "dep-a", "dep-x", "dep-!b"]
+ assert [dep.name for dep in configs['a-x'].deps] == expected
+ expected = ["dep-all", "dep-b", "dep-!a", "dep-!x"]
+ assert [dep.name for dep in configs['b'].deps] == expected
+ expected = ["dep-all", "dep-a", "dep-x", "dep-!b"]
+ assert [dep.name for dep in configs['a-x'].deps] == expected
+ expected = ["dep-all", "dep-b", "dep-!a", "dep-!x"]
+ assert [dep.name for dep in configs['b'].deps] == expected
 
  def test_factor_ops(self, newconfig):
  inisource = """
@@ -1392,16 +1401,25 @@ def test_factor_ops(self, newconfig):
  a,b: dep-a-or-b
  a-x: dep-a-and-x
  {a,b}-y: dep-ab-and-y
+ a-!x: dep-a-and-!x
+ a,!x: dep-a-or-!x
+ !a-!x: dep-!a-and-!x
+ !a,!x: dep-!a-or-!x
+ !a-!b: dep-!a-and-!b
+ !a-!b-!x-!y: dep-!a-and-!b-and-!x-and-!y
  """
  configs = newconfig([], inisource).envconfigs
 
  def get_deps(env):
  return [dep.name for dep in configs[env].deps]
 
- assert get_deps("a-x") == ["dep-a-or-b", "dep-a-and-x"]
- assert get_deps("a-y") == ["dep-a-or-b", "dep-ab-and-y"]
- assert get_deps("b-x") == ["dep-a-or-b"]
- assert get_deps("b-y") == ["dep-a-or-b", "dep-ab-and-y"]
+ assert get_deps("a-x") == ["dep-a-or-b", "dep-a-and-x", "dep-a-or-!x"]
+ assert get_deps("a-y") == ["dep-a-or-b", "dep-ab-and-y",
+ "dep-a-and-!x", "dep-a-or-!x",
+ "dep-!a-or-!x"]
+ assert get_deps("b-x") == ["dep-a-or-b", "dep-!a-or-!x"]
+ assert get_deps("b-y") == ["dep-a-or-b", "dep-ab-and-y", "dep-a-or-!x",
+ "dep-!a-and-!x", "dep-!a-or-!x"]
 
  def test_envconfigs_based_on_factors(self, newconfig):
  inisource = """
@@ -1410,6 +1428,9 @@ def test_envconfigs_based_on_factors(self, newconfig):
  a: something
  b,c: something
  d-e: something
+ !f: something
+ !g,!h: something
+ !i-!j: something
 
  [unknown-section]
  some-setting=
@@ -1424,7 +1445,7 @@ def test_envconfigs_based_on_factors(self, newconfig):
  config = newconfig(["-e py3-spam"], inisource)
  assert not config.envconfigs
  assert config.envlist == ["py3-spam"]
- for x in "abcde":
+ for x in "abcdefghij":
  env = "py3-{}".format(x)
  config = newconfig(["-e {}".format(env)], inisource)
  assert sorted(config.envconfigs) == [env]

tox/config.py

@@ -848,8 +848,8 @@ def _list_section_factors(self, section):
  factors = set()
  if section in self._cfg:
  for _, value in self._cfg[section].items():
- exprs = re.findall(r'^([\w{}\.,-]+)\:\s+', value, re.M)
- factors.update(*mapcat(_split_factor_expr, exprs))
+ exprs = re.findall(r'^([\w{}\.!,-]+)\:\s+', value, re.M)
+ factors.update(*mapcat(_split_factor_expr_all, exprs))
  return factors
 
  def make_envconfig(self, name, section, subs, config, replace=True):
@@ -912,9 +912,31 @@ def _split_env(env):
  return mapcat(_expand_envstr, env)
 
 
+def _is_negated_factor(factor):
+ return factor.startswith('!')
+
+
+def _base_factor_name(factor):
+ return factor[1:] if _is_negated_factor(factor) else factor
+
+
 def _split_factor_expr(expr):
+ def split_single(e):
+ raw = e.split('-')
+ included = {_base_factor_name(factor) for factor in raw
+ if not _is_negated_factor(factor)}
+ excluded = {_base_factor_name(factor) for factor in raw
+ if _is_negated_factor(factor)}
+ return included, excluded
+
+ partial_envs = _expand_envstr(expr)
+ return [split_single(e) for e in partial_envs]
+
+
+def _split_factor_expr_all(expr):
  partial_envs = _expand_envstr(expr)
- return [set(e.split('-')) for e in partial_envs]
+ return [{_base_factor_name(factor) for factor in e.split('-')}
+ for e in partial_envs]
 
 
 def _expand_envstr(envstr):
@@ -1064,12 +1086,14 @@ def getstring(self, name, default=None, replace=True, crossonly=False):
 
  def _apply_factors(self, s):
  def factor_line(line):
- m = re.search(r'^([\w{}\.,-]+)\:\s+(.+)', line)
+ m = re.search(r'^([\w{}\.!,-]+)\:\s+(.+)', line)
  if not m:
  return line
 
  expr, line = m.groups()
- if any(fs <= self.factors for fs in _split_factor_expr(expr)):
+ if any(included <= self.factors
+ and not any(x in self.factors for x in excluded)
+ for included, excluded in _split_factor_expr(expr)):
  return line
 
  lines = s.strip().splitlines()