Taskflow VS tensorflow

Martin, have you had a look at https://github.com/taskflow/taskflow ?

> The set-theory approach is hard to do, but promising. Each object that wants something has a small set of the things it wants. There's a big pool of such sets. There's also a big pool of the items you have, which changes constantly. It's easy to express what you need to fetch and which objects are now ready to go as set intersection and difference operations. But you need representations for big sparse sets which can do set operations fast. Probably B-trees, or something in that space.

Incremental updates to dynamic dependency graphs is a familiar problem for build tooling. I personally have used the taskflow C++ library (https://github.com/taskflow/taskflow) to great effect.

> Microsoft Research fooled around with this concept years ago in a different context. The idea was to have a database which supported pending SQL queries. The query would return new results when the database changed such that the results of the query changed. The goal was to to support that for millions of pending queries. Financial traders would love to have that. It's a very hard scaling problem. Don't know how that came out.

Incremental view maintenance is an active area of research. The likes of Noria and Materialize have done this with SQL, and the pg_ivm Postgres extension looks promising. Not sure if there is an equivalent implementation geared towards entity-component systems, though.

For parallel programming nowadays, personally I reach for C++ Taskflow when I really care about performance, or a mix of core.async and running multiple load balanced instances when I’m doing more traditional web backend stuff in Clojure.

Also, you can take a look to good libraries. The problem is that very often libraries are heavily templated, so It could be hard. For example, I like the style of the Taskflow library, I think is very clear, is relatively small, while makes use of more advanced techniques: https://github.com/taskflow/taskflow

Cool. Thanks! How does it compare to taskflow?

I've not seen yet, but it's been a bit since I looked last, any evidence of being able to build a computation graph and "save" it to re-run on new inputs. Something like https://github.com/taskflow/taskflow

It seems you're describing something a generic parallel task framework. Check taskflow for a production ready example https://github.com/taskflow/taskflow/blob/master/

- https://github.com/taskflow/taskflow (I recommend to learn it first since its API and documentation are excellent)

If you want some sort of "job" system, where you submit items to a some sort of queue to be processed in parallel, try searching for a thread pool - there isn't one in the standard library, but there's about a million implementations online. There are more complicated versions of that idea, that describe computation as a directed acyclic graph, such as taskflow.

Recently, I was working on various open source projects. Mostly I was working on issues labelled as "good first issues". Before I start working on a issue I usually take a look at the previous attempts made by other people. Usually I can find a couple of PRs or comments if the issue is older than one year. Looking at the work history usually it becomes obvious why previous attempts were not successful: someone completely misunderstood the task and went the wrong direction, someone can barely code, someone is just pinging the contributors instead of doing the actual work[1]. What was surprising to me was that the maintainers almost always found time to answer every question, review ridiculously bad and incorrect PRs and spend days communicating with the contributor (including me) to give feedback about the work, while they themselves could absolutely resolve the issue in less than an hour. To be honest I don't understand why someone with 10+ years of experience would spend time reviewing my "good first issue" PRs like they don't have anything else to do. I am trying to be useful and to learn but I don’t want to bother anyone or waste anyone’s time and that's why I am wondering why would maintainers spend their time reviewing my code? What is their motivation?

[1] https://github.com/tensorflow/tensorflow/issues/22926#issuecomment-2888463504

You guys using convoluted git commands when a single line of subversion works:

svn checkout https://github.com/tensorflow/tensorflow/trunk/tensorflow/ex...

The TensorFlow team structures their contributor guide with clear sections and sub-sections, making it easy to jump to relevant information without reading everything.

We see an addition to the AI community with AutoGPT. Along with Tensorflow they represent the AI community in the software category, which is getting relevant (2 out of 8). We can expect in the future to have new AI projects in the top 25 such as Transformers or Ollama (currently top 34 and 36, respectively).

TensorFlow

The project looks fairly active, based on the commit history:

https://github.com/tensorflow/tensorflow/commits/master/

🥷 Fork & clone tensorflow repo. ⚙️ Setup Development environment for contribution. 🫸 Push and create pull request.

Star on GitHub ⭐ - Tensorflow

Access the repository for source code and examples.