Rework of dfs functions to recursively (changing return type of recurse-able functions)

[Jolanrensen]

Same as #361 but using a different tactic.

Largest changes:

Streamlined behavior of all, cols, groups etc. such that if it's called on

• a SingleColumn<*> containing a single ColumnGroup (such as DataFrame), the operation will run on the children of that ColumnGroup
• else, or a normal ColumnSet<*>, it will simply run on the columns inside.
  This is enforced using the new allInternal() function combined with isSingleColumnWithGroup. I think this simply enforces expected behavior, since all tests work with it (and break upon a slight modification of this function) and more importantly, the calls seem to make sense. Let me know what you think.

Introducing TransformableColumnSet and TransformableSingleColumn. A wrapper around the original ones that provide a way to transformResolve as well as the normal resolve. transformResolve takes a ColumnSetTransformer that can get executed before resolving the ColumnSet or SingleColumn. A ColumnSetTransformer contains implementations for both transform and transformSingle, similar to the functions in Utils.kt. Currently, the only implementation of this is RecursivelyTransformer, which is configured and created in the TransformableColumnSet<>.recursively() calls.
Not all ColumnSets are transformable. Functions that allow reconfiguration with a ColumnSetTransformer need to explicitly state that in their return type. In principle, any function that uses transform {} can automatically opt-in to the TransformableColumnSet return-type if that makes sense for them. Currently, I added the ability to cols {}, all(), groups {}, children {}, filter {}, nameContains(), endsWith(), startsWith(), except(), and withoutNulls().
I added TransformableSingleColumn to first {}, last {}, and single {} (which uses singleWithTransformerImpl to pass on the transformer to the right place).
Of course, if other functions are applicable to modification calls like .recursively() I'd gladly hear it.

.recursively() / .rec() is the replacement of dfs {}, dfsOf<>{}, and allDfs(). because the name was confusing. I put deprecations in place and replaced all calls in the library to recursively. Some things to note:

• dfs {}, called on either a ColumnSet or SingleColumn would exclude the top-level of columns but still include column groups in the end-result, so a proper replacement of that call would be cols {}.recursively() if called on a SingleColumn and cols {}.recursively(includeTopLevel = false) if called on a normal ColumnSet. Of course rec() can be used as shorthand.
• dfsOf<>(), is replaced similarly to either colsOf<>().rec() or colsOf<>().rec(includeTopLevel = false).
• allDfs() defaulted includeGroups to false, so be careful, since this is now true. It can be replaced either with all().recursively() or allRecursively() (or allRec()), again taking into account includeTopLevel and includeGroups.

Other smaller changes:

• Renamed occurrences of "Dfs" where "DataFrames" was meant as such, to avoid confusion
• Renamed:
  • inner functions named dfs to depthFirstSearch where depth first search was actually meant
  • TreeNode.dfs {} -> TreeNode.allChildren {} (+ other dfs functions)
  • TreeNode.topDfs {} -> TreeNode.topmostChildren {} (+ other topDfs functions)
  • Iterable<ColumnWithPath<*>>.dfs() -> flattenRecursively()
  • Some unavoidable linting in the files I touched
• Updated korro to 0.1.4
• Removed ColumnPath.something overloads in ColumnsSelectionDsl since they are handled by SingleColumn<*>
• Added docs to ColumnSet and SingleColumn
• Reworked tests for ColumnsSelectionDsl
• Added recursively tests

[zaleslaw]

topmostChildren?
topDFS is a bad name, but topmostChildren leads me to the mist again

[Jolanrensen]

what would you suggest?

[Jolanrensen]

As my doc explains it takes the tree and returns first the children, closest to the root of the tree, where the predicate holds. It doesn't matter whether the children of those nodes also hold, those are ignored. So since we now have allChildren { something == true }, I thought topmostChildren { something == true }. It can also be viewed as subtree-roots where the predicate holds.

[Jolanrensen]

@koperagen Any suggestions? :)

[koperagen]

Why has the default value for includeGroups changed? I would argue that one of the main usages of dfsOf is to convert / replace columns with some type and groups are not needed

[koperagen]

Is it correct that it's because of unification of overloads?

[Jolanrensen]

Yes, by default groups were included in dfs {}, but excluded in allDfs() if I recall correctly. I had to choose a sensible default. But if you use colsOf<Int>().rec(), then includeGroups irrelevant anyways, since Int is not a Group.

[koperagen]

Hmm, this is interesting and a bit confusing. I think this parameter only exists for allDfs. For normal dfs it would be dfs { !it.isColumnGroup() }. Now it's also present for rec, recursively. What if you remove it from them and leave only for allRec with the same default value (true)?

[Jolanrensen]

.rec() can not only be called on all(), you can also call first { }.rec(includeGroups = false), so I think keeping it in rec/recursively too would be best.

[Jolanrensen]

I don't think we have a function that can give all valueColumns right? If we did we could have valueCols {}.recursively() or valueColsRecursively() but that might get a bit long.

[Jolanrensen]

Plus it's not just ColumnGroups that get included right? also FrameColumns and ValueColumns. So, just all of them. I think that's a good default

[Jolanrensen]

@koperagen Actually I do want to add valueCols {} and frameCols {} since we already have groups {} as well. Probably in another PR though. Then you can call valueCols().rec() if you want. Sounds okay?

[koperagen]

Could be a nice addition, not 100% related to this problem though. For this allRecursively function with previous default, something like "allLeafs" might fit better. I think it was named like that before and renamed later

[Jolanrensen]

We could also take it in another direction eventually, similar to colsOf<Type>() have colsOf(vararg ColumnKind) {}.rec(), that might be clearer compared to users having to figure out what a "leaf" means.

[koperagen]

Another thing that i noticed now is that it's hard to tell what rec does for first { it.type == typeOf<Int> }.rec(). Will it give the same result as dfs { it.type == typeOf<Int> }.first()?

[Jolanrensen]

Another thing that i noticed now is that it's hard to tell what rec does for first { it.type == typeOf<Int> }.rec(). Will it give the same result as dfs { it.type == typeOf<Int> }.first()?

Yes, it works the same. You can read it like a sentence: "Select the first column of type Int recursively", which sort of makes sense.
Recursively modifies the call before such that it runs on all columns recursively instead of just on the top-level.

Basically:
allRec().first { it.type == typeOf<Int>() } == first { it.type == typeOf<Int>() }.rec()