|
11 | 11 | * https://docs.spring.io/spring-framework/docs/4.3.15.RELEASE/spring-framework-reference/html/aop.html |
12 | 12 | * https://dotty.epfl.ch/docs/reference/new-types/polymorphic-function-types.html |
13 | 13 |
|
| 14 | +## preface |
| 15 | + |
| 16 | +* task1: create generator of Account |
| 17 | + * then derive it using zio-test/magnolia |
| 18 | +* task2: derive generator of Contributor |
| 19 | + * then switch it to generate Contributor from file `src/test/resources/contributors.txt` |
| 20 | +* task3: create and plug aspect to set specific seed (TestSeed.seed) before each test |
| 21 | + |
14 | 22 | ## aspect oriented programming |
15 | 23 | * lib: caliban |
16 | 24 | * example |
|
133 | 141 | * timeout(60s) @@ repeat(10) |
134 | 142 |
|
135 | 143 | ## property based testing |
136 | | -* example |
| 144 | +* example: ZIO test |
137 | 145 | ``` |
138 | 146 | test("encode and decode is an identity") { |
| 147 | + // check operator, one or more generators, assertion |
139 | 148 | check(genEvents) { event => |
140 | 149 | assert(decode(encode(event)))(equalTo(event)) |
141 | 150 | } |
142 | 151 | } |
143 | 152 | ``` |
144 | | -* support for random and deterministic property based testing |
145 | | -* integrated shrinking |
146 | | -* Property based testing is an approach to testing where the framework generates test cases for us instead of having to come up with test cases ourselves. |
147 | | -* The obvious advantage of property based testing is that it allows us to quickly test a large number of test cases, potentially revealing counterexamples that might not have been obvious. |
148 | | -* Property based tests typically only generate one hundred to two hundred test cases. In contrast, even a single Int can take on more than a billion different values. |
149 | | -* If we are generating more complex data types the number of possibilities increases exponentially |
150 | | - * So in most real world applications of property based testing are only testing a very small portion of the sample space. |
151 | | - * if counterexamples require multiple generated values to take on very specific values then we may not generate an appropriate counterexample even though such a counterexample does exist |
152 | | - * A solution to this is to complement property based testing with traditional tests for particular degenerate cases identified by developers. |
153 | | -* A good generator should also be general enough to generate test cases covering the full range of values over which we expect the property to hold. |
154 | | - * For example, a common mistake would be to test a form that validates user input with a generator of ASCII characters. |
155 | | - * This is probably very natural for many of us to do, but what happens if the user input is in Mandarin? |
156 | | -* In ZIO Test, a property based test always has three parts: |
157 | | - * A check operator |
158 | | - * This tells ZIO Test that we are performing a property based test |
159 | | - * One of more Gen values |
160 | | - * You can think of each Gen as representing a distribution of potential values and each time we run a property based test we sample values from that distribution. |
161 | | - * Generators As Streams Of Samples |
162 | | - ``` |
163 | | - final case class Gen[-R, +A]( |
164 | | - sample: ZStream[R, Nothing, Sample[R, A]] |
165 | | - ) |
166 | | - |
167 | | - final case class Sample[-R, +A]( |
168 | | - value: A, |
169 | | - shrinks: ZStream[R, Nothing, Sample[R, A]] |
170 | | - ) |
171 | | - ``` |
172 | | - * For example, we could imagine a generator that generates values by opening a local file with test data, reading the contents of that file into memory, and each time generating a value based on one of the lines in that file. |
173 | | - * assertion |
174 | | -* https://zio.dev/api/zio/test/magnolia/index.html |
175 | | - * traitDeriveGen[A] extends AnyRef |
176 | | - A DeriveGen[A] can derive a generator of A values. |
177 | | -* To create generators for a data type we will generally follow a two step process. |
178 | | - * First, construct generators for each part of the data type. |
179 | | - * Second, combine these generators using operators on Gen such as flatMap, map, |
180 | | - and oneOf to build a generator for your data type out of these simpler generators. |
181 | | - ``` |
182 | | - final case class Stock(ticker: String, price: Double, currency: Currency) |
183 | | - |
184 | | - lazy val genStock: Gen[Any, Stock] = for { |
185 | | - ticker <- genTicker |
186 | | - price <- genPrice |
187 | | - currency <- genCurrency |
188 | | - } yield Stock(ticker, price, currency) |
189 | | - ``` |
190 | | - * One potential inefficiency you may have noticed in some of the examples above is that the flatMap operator requires us to run our generators sequentially, because the second generator we use can depend on the value generated by the first generator |
191 | | - * ZIO Test supports this through the zipWith and zip operators and their symbolic alias <&> |
192 | | - * These will generate the two values in parallel and then combine them into a tuple or using the specified function. |
193 | | -* This illustrates a helpful principle for working with generators, which is to prefer transforming generators instead of filtering generators. |
194 | | - ``` |
195 | | - val ints: Gen[Random, Int] = Gen.int(1, 100) |
196 | | - val evens: Gen[Random, Int] = ints.map(n => if (n % 2 == 0) n else n + 1) |
197 | | - ``` |
198 | | - * We will see below that we can also filter the values produced by generators, but this has a cost because we have to “throw away” all of the generated data that |
199 | | - doesn’t satisfy our predicate |
200 | | -* The either operator is helpful for when we want to generate data for sum types that can be one type or another, such as the Currency data type above. |
201 | | - ``` |
202 | | - def genTry[R <: Random, A](gen: Gen[R, A]): Gen[R, Try[A]] = Gen.either(Gen.throwable, gen).map { |
203 | | - case Left(e) => Failure(e) |
204 | | - case Right(a) => Success(a) } |
205 | | - ``` |
206 | | - * The first is the oneOf operator, which picks from one of the specified generators with equal probability |
207 | | - * The second is the elements operator, which is like oneOf but just samples from one of a collection of concrete values instead of from one of a collection of generators |
| 153 | +* is an approach where the framework generates test cases |
| 154 | +* advantage: allows to quickly test a large number of test cases |
| 155 | + * potentially: reveal not obvious counterexamples |
| 156 | +* typically generate ~ 100-200 test cases |
| 157 | + * Int ~2 billion values |
| 158 | + * complex data types => number of possibilities increases exponentially |
| 159 | + * complement property based testing with traditional tests (for particular degenerate cases) |
| 160 | +* common mistake: generator is not general enough |
| 161 | + * example: generating user input using ASCII |
| 162 | + * what about: 普通话 ? |
| 163 | +* generator represents a distribution of potential values |
| 164 | + * each time we run a property based test we sample values from that distribution |
208 | 165 | ``` |
209 | | - sealed trait Currency |
210 | | - case object USD extends Currency |
211 | | - case object EUR extends Currency case object JPY extends Currency |
| 166 | + final case class Gen[-R, +A]( |
| 167 | + sample: ZStream[R, Nothing, Sample[R, A]] |
| 168 | + ) |
212 | 169 |
|
213 | | - val genCurrency: Gen[Random, Currency] = Gen.elements(JPY, USD, EUR) |
| 170 | + final case class Sample[-R, +A]( |
| 171 | + value: A, |
| 172 | + shrinks: ZStream[R, Nothing, Sample[R, A]] |
| 173 | + ) |
| 174 | + ``` |
| 175 | +* create generators |
| 176 | + * construct generators for each field |
| 177 | + * combine with operators |
| 178 | + * example: flatMap, map, oneOf, zip |
| 179 | + * example |
| 180 | + ``` |
| 181 | + val genAccount2: Gen[Any, Account] = |
| 182 | + (Gen.uuid <*> // symbolic alias for zip and zipWith; generate values in parallel |
| 183 | + Gen.fromIterable(AccountStatus.values) <*> |
| 184 | + Gen.string1(Gen.char) |
| 185 | + ).map { case (uuid, status, str) => Account(AccountId(uuid), status, NonEmptyString.unsafeFrom(str)) |
| 186 | + } |
| 187 | + ``` |
| 188 | +* auto-deriving generator |
| 189 | + * example |
| 190 | + ``` |
| 191 | + val genAccount: Gen[Any, Account] = DeriveGen[Account] // implicit for each field |
| 192 | + ``` |
| 193 | + * lib: https://zio.dev/api/zio/test/magnolia/index.html |
| 194 | +* don't use filter - transform instead |
| 195 | + * filtering = "throw away" data that doesn’t satisfy our predicate |
| 196 | + * example |
| 197 | + ``` |
| 198 | + val evens: Gen[Random, Int] = ints.map(n => if (n % 2 == 0) n else n + 1) // transformation |
214 | 199 | ``` |
215 | | -* For example, if we wanted to generate a pair of integers we could do it like this: |
216 | | - val pairs: Gen[Random, (Int, Int)] = Gen.int <*> Gen.int |
217 | | -* Random And Deterministic Generators |
218 | | - * Traditionally in property based testing there has been a distinction between random and deterministic property based testing. |
219 | | - * In random property based testing, values are generated using a pseudorandom number generator based on some initial seed. |
220 | | - * The disadvantage of property based testing is that it is impossible for us to ever prove a property with random property based testing, we can merely fail to falsify it. |
221 | | -* Samples And Shrinking |
222 | | - * Typically when we run property based tests the values will be generated randomly and so when we find a counterexample to a property it will typically not be the |
223 | | - “simplest” counterexample. |
224 | | - * To help us it is useful if the test framework tries to shrink failures to ones that are “simpler” in some sense and still violate the property. |
225 | | - * First, we want to generate values that are “smaller” than the original value in some sense. This could mean closer to zero in terms of numbers or closer to zero size in terms of collections. |
226 | | - * Instead of doing this ZIO Test uses a technique called integrated shrinking where every generator already knows how to shrink itself and all operators on generators also appropriately combine the shrinking logic of the original generators. |
227 | | - * So a generator of even integers can’t possibly shrink to anything other than an even integer because it is built that way. |
228 | | - * In addition to a value a Sample also contains a “tree” of possible “shrinkings” of that value. It may not be obvious from the signature but ZStream[R, Nothing, Sample[A]] represents a tree. |
229 | | - * The root of the tree is the original value. |
230 | | - * The next level of the tree consists of all the values for the samples in the shrink collection. |
231 | | - * Each of these values in turn may have its own children, represented by its own shrink tree. |
232 | | - * The shrink tree must obey the following invariants. |
233 | | - * First, within any given level, values to the “left”, that is earlier in the stream, must be “smaller” than values that are later in the stream. |
234 | | - * Second, all children of a value in the tree must be “smaller” than their parents. |
235 | | - * We begin by generating the first Sample in the shrink stream and testing whether its value is also a counterexample to the property being tested. |
236 | | - * If it is a valid counterexample, we recurse on that sample. If it is not, we repeat the process with the next Sample in the original shrink stream. |
237 | | - * For example, the default shrinking logic for integral values first tries to shrink to zero, then to half the distance between the value and zero, then to half that distance, and so on. At each level we repeat the same logic. |
238 | | - * Sample itself has its own operators such as map and flatMap for combining Sample values. |
239 | | - * The map/flatMap operator conceptually transforms both the value of the sample as well as all of its potential shrinkings with the specified function |
240 | | - * example |
241 | | - * map to transform a generator to generate only even integers we are guaranteed that the shrinkings will also contain only even integers. |
| 200 | +* shrinking |
| 201 | + * counterexample will typically not be the "simplest" |
| 202 | + * test framework tries to shrink failures to ones that |
| 203 | + * are "simpler" (in some sense) |
| 204 | + * example: smaller integers, smaller collections |
| 205 | + * and still violate the property |
| 206 | + * ZIO Test uses "integrated shrinking" |
| 207 | + * every generator already knows how to shrink itself |
| 208 | + * all operators keep this property |
| 209 | + * example: generator of even integers can’t shrink to 1 |
| 210 | + * under the hood |
| 211 | + * Sample contains a "tree" of possible "shrinkings" for the value |
| 212 | + * root: original value |
| 213 | + * invariants |
| 214 | + * any given level: value earlier in the stream, must be "smaller" than later values |
| 215 | + * all children must be "smaller" than their parents |
| 216 | + * machinery |
| 217 | + 1. generate the first Sample in the shrink stream |
| 218 | + 1. test whether its value is also a counterexample to the property being tested |
| 219 | + * counterexample => recurse on that sample |
| 220 | + * not => repeat with the next Sample in shrink stream |
| 221 | + * example: shrinking logic for int |
| 222 | + * first tries to shrink to zero |
| 223 | + * then to half the distance between counterexample and zero |
| 224 | + * then to half that distance, and so on |
242 | 225 |
|
243 | 226 | ## seed |
244 | 227 | * TestRandom service provides a testable implementation of the Random service. |
|
0 commit comments