Taskflow VS rust

Martin, have you had a look at https://github.com/taskflow/taskflow ?

> The set-theory approach is hard to do, but promising. Each object that wants something has a small set of the things it wants. There's a big pool of such sets. There's also a big pool of the items you have, which changes constantly. It's easy to express what you need to fetch and which objects are now ready to go as set intersection and difference operations. But you need representations for big sparse sets which can do set operations fast. Probably B-trees, or something in that space.

Incremental updates to dynamic dependency graphs is a familiar problem for build tooling. I personally have used the taskflow C++ library (https://github.com/taskflow/taskflow) to great effect.

> Microsoft Research fooled around with this concept years ago in a different context. The idea was to have a database which supported pending SQL queries. The query would return new results when the database changed such that the results of the query changed. The goal was to to support that for millions of pending queries. Financial traders would love to have that. It's a very hard scaling problem. Don't know how that came out.

Incremental view maintenance is an active area of research. The likes of Noria and Materialize have done this with SQL, and the pg_ivm Postgres extension looks promising. Not sure if there is an equivalent implementation geared towards entity-component systems, though.

For parallel programming nowadays, personally I reach for C++ Taskflow when I really care about performance, or a mix of core.async and running multiple load balanced instances when I’m doing more traditional web backend stuff in Clojure.

Also, you can take a look to good libraries. The problem is that very often libraries are heavily templated, so It could be hard. For example, I like the style of the Taskflow library, I think is very clear, is relatively small, while makes use of more advanced techniques: https://github.com/taskflow/taskflow

Cool. Thanks! How does it compare to taskflow?

I've not seen yet, but it's been a bit since I looked last, any evidence of being able to build a computation graph and "save" it to re-run on new inputs. Something like https://github.com/taskflow/taskflow

It seems you're describing something a generic parallel task framework. Check taskflow for a production ready example https://github.com/taskflow/taskflow/blob/master/

- https://github.com/taskflow/taskflow (I recommend to learn it first since its API and documentation are excellent)

If you want some sort of "job" system, where you submit items to a some sort of queue to be processed in parallel, try searching for a thread pool - there isn't one in the standard library, but there's about a million implementations online. There are more complicated versions of that idea, that describe computation as a directed acyclic graph, such as taskflow.

Funnily this is satire/educational code - notice the license is "GLWTPL" (Good Luck With That Public License).

Repo is based on exploiting this bug - https://github.com/rust-lang/rust/issues/25860 (Open since 2015)

The bug allows you to "extend" any lifetime 'a to 'static in safe code:

This converts a reference with any lifetime into a 'static reference, which violates Rust's safety guarantees. Once you can do this, you can create dangling references, access freed memory, etc.

This is a type system bug, not just an implementation bug. From the discussions:

- The easy fix (banning contravariance in function arguments) would eliminate valid Rust programs

- The proper fix requires changes to how higher-ranked lifetime variables carry subtyping relations

- The Rust team chose to pursue the more rigorous solution as part of replacing the trait solver (the "Chalk" project)

I just created a ticket to better gate that suggestion so it doesn't trigger here[3].

But the second suggestion is exactly what the user intended. The only thing missing would be for the message to actually mention "that Arc::clone() creates a new owned handle that can be moved into the closure independently" as part of the message. That gives makes me think we should have something like `#[diagnostic::on_move("message")]`, so I filed a ticket for that too[4]. At worst we could hardcode better messages for std types.

I am not impressed with the rest of the article, neither the tone nor the substance. I wouldn't have commented if it hadn't been due to the combination of the title with that specific wording (given that I've spent a decade making rustc errors teach you things), the actionable feedback being buried under layers of snark and indirection, and the actual feedback being given being wrong (but hey, at least I did find two other tangential things that could do with fixing, so small wins I guess?).

[1]: https://play.rust-lang.org/?version=stable&mode=debug&editio...

[2]: https://github.com/rust-lang/rust/pull/121652

[3]: https://github.com/rust-lang/rust/issues/149861

[4]: https://github.com/rust-lang/rust/issues/149862

There are plans to experiment with language level comptime support too: https://github.com/rust-lang/rust/pull/148820

SAFETY comments do not magically make unsafe Rust correct nor safe. And Miri cannot catch everything, and is magnitudes slower than regular program running.

https://github.com/rust-lang/rust/commit/71f5cfb21f3fd2f1740...

https://materialize.com/blog/rust-concurrency-bug-unbounded-...

format_args! is slow rust/#76490

> The main benefit of parking lot is that it makes locks very small, which then encourages the use of fine grained locking. For example, in JavaScriptCore (ParkingLot’s first customer), we stuff a 2-bit lock into every object header - so if there is ever a need to do some locking for internal VM reasons on any object we can do that without increasing the size of the object

IMHO that's a very cool feature which is essentially wasted when using it as a `Mutex` because the mutex's size will get rounded up to the alignment of `InnerBlah`. And even when not doing that, afaict `parking_lot` doesn't expose a way to use the remaining six bits in `parking_lot::RawMutex`. I think the new std mutexes made the right choice to use a different design.

> I’m surprised that this only compared to std on one platform (Linux).

Can't speak for the author, but I suspect a lot of people really only care about performance under Linux. I write software that I often develop from a Mac but almost entirely deploy on Linux. (But speaking of Macs: std::mutex doesn't yet use futexes on macOS. Might happen soon. https://github.com/rust-lang/rust/pull/122408)