Ruby

I think the workaround using a variable set after yield (success = true) is not too bad, and clear enough. And the performance would be fine in this case (i.e. there would be no overhead if the JIT profiles the branch and only one branch is taken, like TruffleRuby).

Ensure should always be executed no matter the circumstances IMHO, so I don't like to make it conditional with extra syntax.
Not executing ensure when using break, etc sounds like a bug in the vast majority of cases.

@Eregon (Benoit Daloze) - thanks for your response.

Ensure should always be executed no matter the circumstances IMHO, so I don't like to make it conditional with extra syntax.

This isn't about not executing ensure, but allowing user to handle situations like "returned normally" vs "non-standard flow control".

I think the workaround using a variable set after yield (success = true) is not too bad, and clear enough. And the performance would be fine in this case (i.e. there would be no overhead if the JIT profiles the branch and only one branch is taken, like TruffleRuby).

Personally, I think it's ugly, and I also think it is inefficient. I also don't think it's clearly conveying what the user is trying to do.

The problem is, people write code like this:

begin .. ensure abnormal_path if $! end 

This actually wrong and fails in the case of throw wrapped in catch as given in the original examples.

It's actually not clear from the code if this is what the user wanted or not. Because you can't express clearly what you are actually interested in.

There should be no overhead when exiting normally in the following situation:

begin yield ensure when not return return :abnormal end 

As soon as you write code like:

begin yield success = true ensure return :abnormal unless success end 

you guarantee that the code must pass through the ensure block. But your intention is, only execute this code if the code didn't return normally (or some other flow control).

So, I don't think the argument about always executing ensure holds up - this isn't about changing the semantics of ensure but extending it to handle explicitly what people are already doing, albeit probably incorrectly and inefficiently.

I was about to write a long comment, but then I noticed that my reply would be too long, so I shortened
this. So just a few key statements:

I don't like the proposed syntax suggestion here in particular:

ensure when not return 

The reason why I dislike it is partially due to the syntax verbosity; but also because, more significantly,
because it appears to do conditional checks in very different means than have existed before in ruby.
I would expect "when" usually in a case-structure.

There was this other recent suggestion to also add "when", to allow for chaining after the addition of
the alias "then" to "yield_self", leading to chains of .when .then .when .then; and I also am not a huge fan
of using "when" that way. Of course there is more than one way to do something, but my own personal
preference would be to keep "when" reserved for case-when layouts.

As for catch/throw: I think it is a lot more common to see begin/rescue than catch/throw. Even if this
may be slower (e. g. some people doing control flow logic with begin/rescue), I think the intent and
clarity of intent is much clearer with control flows based on begin/rescue (and ensure) style layout
of code. So to conclude and finish my comment, I think it would be better to not introduce a more
complicated ensure conditional clause as proposed in the suggestion here. IMO you can re-structure
your code to keep ensure simple, so it should not be necessary to add additional conditionals into
ensure, even more so when these are unusual in regards to syntax - it would be the first time that I
would see "ensure when" and I don't quite like that syntax suggestion in this context.

@shevegen - thanks for your feedback.

The reason why I dislike it is partially due to the syntax verbosity

With regards to syntax, it should use existing keywords, otherwise it can cause more problems. The reason for not is already explained, because otherwise user may have to enumerate all other tags which is inconvenient and may break if new tags are introduced in the future. If you can propose better syntax, please do.

but also because, more significantly, because it appears to do conditional checks in very different means than have existed before in ruby.

This isn't really a conditional check like if, and it's not even a big change internally, it simply exposes a bit more of the exception handling mechanisms. It should be more efficient, because ensure when not return can be handled as part of exceptional flow control.

It's more similar to rescue than ensure in it's overhead. In MRI, rescue has some overhead, but it's possible to implement zero-cost exception handling.

The reason why I think this is a good idea is because Ruby provides a lot of flow control statements, but handling these abnormal flow control is very tricky and easy to get wrong. Statements like ensure ... unless $! is actually incorrect as well as inefficient. rescue Exception doesn't handle all ways a function can exit.

I think the key things to think about would be:

• Is there a better syntax for this proposal?
• Does introducing such feature have any practical downside?
  • Implementation cost.
  • Maintenance cost.
  • Does it introduce technical debt?
• Does it solve a problem that can't be solved some other way?
  • ensure when not return can be partly solved some other way but with some overhead.
  • ensure when raise is similar to rescue Exception but doesn't require raise to resume default exception handling.
  • ensure when retry and other formulations (throw, return, next, break, redo, retry) have no equivalent right now.
  • ensure when ..., ..., ... is practically impossible, each case needs to be handled independently if possible.
• Even if there are existing solutions, does this proposed solution communicate more clearly the intention of the programmer?
• Do we want to expose this level of behaviour to user code?
• Could we limit what tags you can match? I suspect the most important one would be ensure when not return, but I can also imagine it could be pretty useful to catch things like retry and redo.

One more thing to bear in mind is that you can avoid this feature if you don't like it. Because it's for some very specific situations. It doesn't affect existing code in any way. But for situations like mine where I have a handful of use cases, and efficiency is a concern (to an extent) as well as readability and correctness, I would welcome such a feature. If you don't need it you won't be affected by it, but if you need it, there is actually no other practical solution in the language right now.

ioquatix (Samuel Williams) wrote:

One more thing to bear in mind is that you can avoid this feature if you don't like it. Because it's for some very specific situations. It doesn't affect existing code in any way. But for situations like mine where I have a handful of use cases, and efficiency is a concern (to an extent) as well as readability and correctness, I would welcome such a feature. If you don't need it you won't be affected by it, but if you need it, there is actually no other practical solution in the language right now.

In the very rare cases where you want to treat a non-local, non-exception exit differently from a normal exit, as Eregon mentioned, there is a simple existing practical solution that is unlikely to be significantly less efficient:

def doot ret = yield normal_exit = true ret ensure # Did the block return normally return "abnormal" if $! || !normal_exit end 

I think we should definitely not add syntax in an attempt to make it easier to treat a non-local, non-exception exit differently than a normal exit, as doing so is usually a mistake.

You may not consider the above approach a simple and practical solution. However, considering you have already conceded that your proposal is for "very specific situations" and you only have "handful of use cases", even if you consider it neither simple nor practical, it's still easy and possible. New syntax is not necessary to support what you want, and I do not think new syntax should be added for "very specific situations" with only a "handful of use cases" when it is already possible to get the behavior you want with existing syntax.

So, @jeremyevans0 (Jeremy Evans), I agree with everything you say.

That being said, this proposal goes beyond just that single case. For example, there is no easy way to capture retry or redo exiting a block.

Looking at your example, e.g.

def doot ret = yield normal_exit = true ret ensure # Did the block return normally return "abnormal" if $! || !normal_exit end 

I certainly prefer

def doot yield ensure when not return return "abnormal" end 

It's clearer what's going on (especially if function is more complex than 1 line of actual code), it should be more efficiently implemented by VM since it only affects non-normal flow control and there is no conditional checks required.

ioquatix (Samuel Williams) wrote:

For example, there is no easy way to capture retry or redo exiting a block.

Which is intended, it would break the semantics of those if they were caught, or it would be very surprising if some ensure blocks are not run in these cases even though they exit the scope.

I think there is < 1% case where it's sensible to not run ensure in this kind of conditions, and the workaround seems good enough for these rare cases.

it should be more efficiently implemented by VM since it only affects non-normal flow control and there is no conditional checks required.

Please don't assume that. A check will be needed too in the general case, and it can already optimize very well with the local variable (e.g., zero overhead on TruffleRuby if only using the normal path, or only the exception path).

@ioquatix (Samuel Williams) Could you give one or two real-world examples where this would be useful?

Okay, thanks for your time and input. That makes sense.

I revisited this situation again.

This behaviour is very confusing:

def transaction begin puts "Begin Transaction" yield rescue puts "Abort Transaction" else puts "Commit Transaction" end end catch(:ball) do transaction do throw :ball end end 

In particular, trying to handle this by using ensure ... unless $! can fail in some situations due to scope of $!

def transaction begin puts "Begin Transaction" yield rescue puts "Abort Transaction" ensure puts "Commit Transaction" unless $! end end catch(:ball) do begin raise "Problem" rescue transaction do throw :ball end end end 

It seems like we must involve some elaborate state tracking in order to get the correct behaviour (calling either abort or commit predictably). So, I believe we should reconsider this feature, or some evolution of it.

Here is working example:

def transaction failed = false begin puts "Begin Transaction" yield rescue Exception puts "Abort Transaction" failed = true raise ensure puts "Commit Transaction" unless failed end end catch(:ball) do begin raise "Problem" rescue transaction do throw :ball end end end 

This seems overly complex to me personally.

ioquatix (Samuel Williams) wrote in #note-14:

Here is working example:

def transaction failed = false begin puts "Begin Transaction" yield rescue Exception puts "Abort Transaction" failed = true raise ensure puts "Commit Transaction" unless failed end end catch(:ball) do begin raise "Problem" rescue transaction do throw :ball end end end 

This seems overly complex to me personally.

You can simplify this slightly using a local variable:

def transaction failed = false begin puts "Begin Transaction" yield rescue Exception => exc puts "Abort Transaction" raise ensure puts "Commit Transaction" unless exc end end catch(:ball) do begin raise "Problem" rescue transaction do throw :ball end end end 

I think such code makes sense. Exceptions abort the transaction, but should be reraised and not swallowed, so the raise doesn't feel out of place.

Looking at my earlier example:

def doot ret = yield normal_exit = true ret ensure # Did the block return normally return "abnormal" if $! || !normal_exit end 

This is a case that wouldn't work correctly in the transaction scenario, where it is called inside an existing rescue block. So you would currently have to use:

def doot ret = yield normal_exit = true ret rescue Exception => exc raise ensure # Did the block return normally return "abnormal" if exc || !normal_exit end 

About the only way I can think to make that easier would be for ensure to support =>:

def doot ret = yield normal_exit = true ret ensure => exc # Did the block return normally return "abnormal" if exc || !normal_exit end 

In this case exc would be nil if the code before ensure did not raise an exception, and the exception instance if it did.

However, I think the cases where this actually matters (ensure without rescue but that cares about whether an exception has been raised) are so few that we shouldn't consider language modifications to support them.

What's wrong with this, which works fine?

def transaction begin puts "Begin Transaction" result = yield success = true result ensure if success puts "Commit Transaction" else puts "Abort Transaction" end end end catch(:ball) do transaction do throw :ball end end 

IMHO it's clear and explicit.
What we want to check here is ultimately if the block was successful (no error of any sort) and the local variable is pretty clear.
In fact, you might even want to allow the block to return :abort or so, and then it would simply be if success && result != :abort.

FWIW I think there are very few use cases where throw/catch are a good idea, but this applies to break and others.
Short answer: use ensure if you want to be sure something runs, rescue only captures Exception, as expected.

I suspect that there is existing buggy code which has ensure blocks and checks $! which fails in the case I mentioned.

I think the ensure block should be able to capture the current exception without using a global.

I like the proposed syntax:

begin ... ensure => exception if exception abort else commit end end 

It allows existing code that uses if $! to be retrofitted to do the correct thing.

@Eregon (Benoit Daloze) throw should not abort the transaction, it should be committed.

I suspect that there is existing buggy code which has ensure blocks and checks $! which fails in the case I mentioned.

Typically that's an anti-pattern.
In 99% cases, the ensure code should do its logic regardless whether there is an ongoing exception/unwind or not.

Seems Jeremy agrees in comment 7:

I think we should definitely not add syntax in an attempt to make it easier to treat a non-local, non-exception exit differently than a normal exit, as doing so is usually a mistake.

Eregon (Benoit Daloze) throw should not abort the transaction, it should be committed.

That is surprising to me.
If throw is performed in the middle of the transaction block then maybe only half the operations are done, committing in that case seems wrong.

transaction do update1 some_call_that_ends_up_in_throw update2 end 

Eregon (Benoit Daloze) wrote in #note-19:

Eregon (Benoit Daloze) throw should not abort the transaction, it should be committed.

That is surprising to me.
If throw is performed in the middle of the transaction block then maybe only half the operations are done, committing in that case seems wrong.

transaction do update1 some_call_that_ends_up_in_throw update2 end 

Rails 7 is going to start treating non-local, non-exception exits as rolling back the transaction, the same as exception exits. I believe this is to work around the fact that Timeout uses throw (https://github.com/rails/rails/pull/29333).

Sequel won't make the same change. I think that non-local, non-exception exits should commit the transaction, and only exceptions should roll the transaction back. Changing the behavior just because of Timeout's design (even if Timeout is not used) seems to be a mistake to me. I certainly have a lot of code that uses throw to exit transactions and expects the transaction will be committed. Sinatra and Roda both use throw for returning responses to web requests, for example:

DB.transaction do DB[:table].insert unless update_related_table? redirect '/some-path' # throw end DB[:related_table].update end 

Thanks for the link.

Is throw used for other things than redirect?

If redirect would use a regular exception, and for instance pass [] as the backtrace, it would probably be as efficient, but then gives the possibility to differentiate for example with rescue => e; e&.should_commit? or so.
throw/catch has AFAIK no way to communicate any information, so there is no way to know if the throw is meant as a convenience return like for redirect or as some kind of bailout which should not commit the transaction.
It can't even be differentiated from return/break so it seems a bad idea to use throw in the first place.

It feels weird that Timeout uses throw and not raise, and it seems counter-intuitive (everyone knows Timeout.timeout {} raises Timeout::Error, and exception, throw is unexpected)
It also adds quite some complexity: https://github.com/ruby/timeout/blob/4893cde0eda321448a1a86487ac9b571f6c35727/lib/timeout.rb#L29-L50
Does anyone know what's the point of that?

What I could find is https://github.com/ruby/timeout/commit/238c003c921e6e555760f8e96968562a622a99c4
and https://bugs.ruby-lang.org/issues/8730

IMHO rescue Exception without re-raise is always the bug.

Eregon (Benoit Daloze) wrote in #note-21:

Thanks for the link.

Is throw used for other things than redirect?

Yes, it's used anytime you want to return a response immediately. I just used redirect as an example.

If redirect would use a regular exception, and for instance pass [] as the backtrace, it would probably be as efficient, but then gives the possibility to differentiate for example with rescue => e; e&.should_commit? or so.
throw/catch has AFAIK no way to communicate any information, so there is no way to know if the throw is meant as a convenience return like for redirect or as some kind of bailout which should not commit the transaction.
It can't even be differentiated from return/break so it seems a bad idea to use throw in the first place.

Changing a throw (which doesn't indicate error) to an exception (which indicates error) for a case that is not an error makes no sense to me. The problem is that Timeout uses throw instead of an exception for error cases (timeouts), not code that uses throw for non-error cases.

It feels weird that Timeout uses throw and not raise, and it seems counter-intuitive (everyone knows Timeout.timeout {} raises Timeout::Error, and exception, throw is unexpected)
It also adds quite some complexity: https://github.com/ruby/timeout/blob/4893cde0eda321448a1a86487ac9b571f6c35727/lib/timeout.rb#L29-L50
Does anyone know what's the point of that?

I believe this is because exception handling inside the Timeout.timeout block doesn't interfere with timeout handling.

What I could find is https://github.com/ruby/timeout/commit/238c003c921e6e555760f8e96968562a622a99c4
and https://bugs.ruby-lang.org/issues/8730

Yep, that pretty much confirms it. I don't agree with the tradeoff. Using throw for error cases is wrong, IMO. It would be better to have cases where Timeout didn't work due to improper rescuing, than not being able to use throw correctly because of Timeout's implementation. There are already places where Timeout doesn't work correctly, after all. A timeout error in my mind is more similar to a TERM signal, and Ruby handles that using an exception.

I should point out that this is not just a throw issue, it occurs for any non local exit. Here's an example using return. We have to use a transaction around the initial retrieval of the record due to the use of FOR UPDATE, because in certain conditions we want to update the record later, and don't want any modifications to the record in between. We need the transaction to commit and not rollback on exit, otherwise we lose the update to the associated object.

def foo record = nil transaction do return :a unless record = dataset.for_update.first record.associated_object.update() return :b if record.some_condition? record.update() if record.some_other_condition? end record end 

While you can generally rewrite code to avoid non-local exits like this, often the non-local exit approach is simplest.

A timeout error in my mind is more similar to a TERM signal, and Ruby handles that using an exception.

I think that's a good point.
I agree for similar things like Ctrl+C/SIGINT which is an Interrupt exception (< SignalException < Exception), and Kernel#exit which is a SystemExit exception (< Exception).

Nothing can guarantee to bubble through user code without stopping in between (it's always possible to hang/loop via ensure),
except a fatal error that would immediately end the process or skip even ensure blocks and always reach the C main().
So proper exception handling (e.g. no rescue Exception; # ignore, ensure code must be bounded in time/steps) is always needed for correctness, and if Ctrl+C/exit work with an exception, so should Timeout.timeout.
Maybe we should open a new issue so that Timeout.timeout uses an Exception and not throw?

I'm not sure if throw is always used as non-local control flow, and not as some sort of exception (e.g., to indicate some 403).

BTW I guess we could replace throw with a non-local return using handle_request(proc { return }) instead of catch, but it wouldn't change anything related to this issue, both are considered non-local exits and non-distinguishable.
That's an argument for treating throw as control flow much like non-local return and not as exceptional though.

Thanks for the discussion. I'll try to summarise my position briefly.

Maybe we should open a new issue so that Timeout.timeout uses an Exception and not throw?

I agree Timeout should raise an exception internally and we should change this. Happy to work on a PR with you all. I am very strongly in favour of this. If we decide to do this, we should let Rails team know about our intended change so they don't end up working themselves into a corner.

Yes, it's used anytime you want to return a response immediately. I just used redirect as an example.

Yes, I also do this and agree with Jeremy's position here. While I do respect everyone's input, to me, exception = abort transaction, throw/break/next = commit.

Therefore, I do feel strongly that we need some precise way to capture the exception as we go through the ensure block. The previous suggestion makes sense to me, as in:

begin raise "Boom" ensure => exception end 

This actually makes it very easy to do the correct thing. The alternatives are tricky to get right:

failed = false begin rescue Exception # "Exception" usually missing... so buggy. # error failed = true raise ensure if failed abort else commit end end 

I'm pretty sure I've got bugs in my code due to this, due to mis-understanding the flow control. So, I think we should make this easier for developers to get right.

Here are some examples of situations where $! is used inside ensure block. In every case, there is a potential buggy behaviour.

https://github.com/drbrain/net-http-persistent/blob/75574f2546a08aa2663b06a2e005bcf2ee304f13/lib/net/http/persistent.rb#L230

https://github.com/copiousfreetime/launchy/blob/80e723ebf2d9e73d35ae5de31a73268eb6c46ecc/lib/launchy.rb#L36-L40

https://github.com/rails/spring/blob/577cf01f232bb6dbd0ade7df2df2ac209697e741/lib/spring/application.rb#L296-L301

https://github.com/ahoward/open4/blob/2bc378285dcec9c88613fd4def57a58606c0680b/lib/open4.rb#L189-L192

I can find more, but I think it's sufficient to show the problem.

Also there is a problem with the proposed work around:

begin rescue Exception => exception raise ensure if exception abort else commit end end 

This only works provided you don't have any other rescue blocks, which you can see in some of the above examples may be an issue, or at least require careful implementation.