Skip to content

Commit d71aa8a

Browse files
committed
Stop retrying semaphore decrement when getting a deadlock
This was added in d72fe46 and initially, it seemed like a good idea, but it ended up causing a different error as it messed with Active Record's transactions. The error would be ``` ActiveRecord::StatementInvalid (Mysql2::Error: SAVEPOINT active_record_1 does not exist) ``` And this made us realise it might be more complicated, because we might be in a transaction that wraps the Solid Queue transaction. Since this is such an edge case that happens only with certain job loads and perhaps only with MySQL, InnoDB and READ COMMITTED, let's stop trying to be too clever about it, and instead, simplify this a bit by avoiding the decrement at all when the semaphore's limit is 1 and it was just created.
1 parent 5b8577d commit d71aa8a

File tree

1 file changed

+5
-22
lines changed

1 file changed

+5
-22
lines changed

app/models/solid_queue/semaphore.rb

+5-22
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,6 @@ def self.signal_all(jobs)
2626

2727
def initialize(job)
2828
@job = job
29-
@retries = 0
3029
end
3130

3231
def wait
@@ -42,42 +41,26 @@ def signal
4241
end
4342

4443
private
45-
attr_accessor :job, :retries
44+
attr_accessor :job
4645

4746
def attempt_creation
4847
Semaphore.create!(key: key, value: limit - 1, expires_at: expires_at)
4948
true
5049
rescue ActiveRecord::RecordNotUnique
51-
attempt_decrement
50+
if limit == 1 then false
51+
else
52+
attempt_decrement
53+
end
5254
end
5355

5456
def attempt_decrement
5557
Semaphore.available.where(key: key).update_all([ "value = value - 1, expires_at = ?", expires_at ]) > 0
56-
rescue ActiveRecord::Deadlocked
57-
if retriable? then attempt_retry
58-
else
59-
raise
60-
end
6158
end
6259

6360
def attempt_increment
6461
Semaphore.where(key: key, value: ...limit).update_all([ "value = value + 1, expires_at = ?", expires_at ]) > 0
6562
end
6663

67-
def attempt_retry
68-
self.retries += 1
69-
70-
if semaphore = Semaphore.find_by(key: key)
71-
semaphore.value > 0 && attempt_decrement
72-
end
73-
end
74-
75-
MAX_RETRIES = 1
76-
77-
def retriable?
78-
retries < MAX_RETRIES
79-
end
80-
8164
def key
8265
job.concurrency_key
8366
end

0 commit comments

Comments
 (0)