Handle concurrent request by waiting the result of an already running operation I need to handle concurrent request by waiting the result of an already running operation.

Requests for data may come in simultaneously with same/different credentials.

For each unique set of credentials there can be at most one GetDataInternal call in progress, with the result from that one call returned to all queued waiters when it is ready.

After this the results of the previous call are invalidated, and a new GetDataInternal call will be allowed with the same set of credentials.

Parallel calls to GetDataInternal with different credentials are allowed.

private readonly ConcurrentDictionary&lt;Credential, Lazy&lt;Data&gt;&gt; Cache
= new ConcurrentDictionary&lt;Credential, Lazy&lt;Data&gt;&gt;();

public Data GetData(Credential credential)
{
    // This instance will be thrown away if a cached
    // value with our "credential" key already exists.
    Lazy&lt;Data&gt; newLazy = new Lazy&lt;Data&gt;(
        () =&gt; GetDataInternal(credential),
        LazyThreadSafetyMode.ExecutionAndPublication
    );

    Lazy&lt;Data&gt; lazy = Cache.GetOrAdd(credential, newLazy);
    bool added = ReferenceEquals(newLazy, lazy); // If true, we won the race.
    Data data;

    try
    {
       // Wait for the GetDataInternal call to complete.
       data = lazy.Value;
    }
    finally
    {
        // Only the thread which created the cache value
        // is allowed to remove it, to prevent races.
        if (added) {
            Cache.TryRemove(credential, out lazy);
        }
    }

    return data;
}


Do I have a bugs? Is it thread safe? How can I improve my code? I'm using .net framework 4.5.2
Reading through the code, I can say that it is well written, concurrent access in mind, and at the first sight, it can be said that it has no flaws.

I especially liked these lines:

Lazy&lt;Data&gt; lazy = Cache.GetOrAdd(credential, newLazy);
bool added = ReferenceEquals(newLazy, lazy); // If true, we won the race.


To prove that it is properly working, I run a 100 concurrent threads test with random access to the GetData() operation with the same credential instance. This test revealed that,


The access is synchronized to the GetDataInternal() method. OK.
For the threads that call GetData() at the same time passing the same Credential instance, after the call to GetOrAdd(), added is true for only the first invoking thread. OK.
All other threads get the existing Lazy instance after calling GetOrAdd() and added is false. OK
GetDataInternal() is called on one of the first threads that call GetOrAdd(), most of the time the adding thread, but sometimes on another. OK
After the call to GetDataInternal(), all requesting threads get the same Data instance out of lazy.Value one by one, as returned by GetDataInternal(). OK
The first thread that gets the Data instance calls TryRemove() and succeeds. OK.
After the Lazy instance is removed, other threads continue to get the old Data instance because they have the removed Lazy instance. OK


No concurrency issues, nothing.

The code is working properly.

ORIGINAL ORDER  THREAD ID   METHOD  EVENT NAME  RESULT  TOTAL MS
2   T:8 M:GetData   E:end_call_GetOrAdd()-&gt;added    R:True  4,82
1   T:9 M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 5,09
12  T:21    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 5,3
9   T:30    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 5,32
13  T:3 M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 5,33
49  T:31    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 5,56
20  T:22    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 6,07
11  T:7 M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 6,19
8   T:18    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 6,41
3   T:23    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 7,01
4   T:10    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 7,29
5   T:6 M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 7,29
15  T:28    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 8,39
70  T:24    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 9,26
10  T:17    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 9,29
45  T:13    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 9,31
18  T:20    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 9,72
7   T:25    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 10,38
40  T:26    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 10,4
22  T:14    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 10,6
65  T:15    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 10,67
52  T:12    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 11,04
47  T:5 M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 11,5
42  T:19    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 11,81
51  T:4 M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 12,34
33  T:11    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 12,91
27  T:29    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 13,62
36  T:27    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 14,01
6   T:16    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 14,88
14  T:9 M:GetDataInternal   E:Return    R:[Data_1]  18,21
17  T:9 M:GetData   E:end_get_lazy.Value    R:[Data_1]  18,29
16  T:28    M:GetData   E:end_get_lazy.Value    R:[Data_1]  18,33
19  T:20    M:GetData   E:end_get_lazy.Value    R:[Data_1]  18,58
21  T:22    M:GetData   E:end_get_lazy.Value    R:[Data_1]  18,71
23  T:14    M:GetData   E:end_get_lazy.Value    R:[Data_1]  18,86
24  T:8 M:GetData   E:end_get_lazy.Value    R:[Data_1]  18,91
25  T:8 M:GetData   E:begin_call_TryRemove  R:- 18,96
50  T:25    M:GetData   E:end_get_lazy.Value    R:[Data_1]  19,83
26  T:8 M:GetData   E:end_call_TryRemove    R:True  19,86
39  T:10    M:GetData   E:end_get_lazy.Value    R:[Data_1]  20,67
38  T:6 M:GetData   E:end_get_lazy.Value    R:[Data_1]  21,56
31  T:17    M:GetData   E:end_get_lazy.Value    R:[Data_1]  21,6
28  T:29    M:GetData   E:end_get_lazy.Value    R:[Data_1]  22
29  T:32    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:True  22,46
53  T:18    M:GetData   E:end_get_lazy.Value    R:[Data_1]  22,61
30  T:21    M:GetData   E:end_get_lazy.Value    R:[Data_1]  24,47
32  T:30    M:GetData   E:end_get_lazy.Value    R:[Data_1]  24,91
35  T:3 M:GetData   E:end_get_lazy.Value    R:[Data_1]  25,13
34  T:11    M:GetData   E:end_get_lazy.Value    R:[Data_1]  25,78
37  T:27    M:GetData   E:end_get_lazy.Value    R:[Data_1]  27,08
41  T:26    M:GetData   E:end_get_lazy.Value    R:[Data_1]  30,53
43  T:19    M:GetData   E:end_get_lazy.Value    R:[Data_1]  31,13
44  T:33    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 31,3
46  T:13    M:GetData   E:end_get_lazy.Value    R:[Data_1]  32,4
48  T:5 M:GetData   E:end_get_lazy.Value    R:[Data_1]  33,04
54  T:23    M:GetData   E:end_get_lazy.Value    R:[Data_1]  33,16
64  T:16    M:GetData   E:end_get_lazy.Value    R:[Data_1]  33,7
56  T:31    M:GetData   E:end_get_lazy.Value    R:[Data_1]  33,96
58  T:7 M:GetData   E:end_get_lazy.Value    R:[Data_1]  34,27
59  T:32    M:GetDataInternal   E:Return    R:[Data_2]  34,65
55  T:4 M:GetData   E:end_get_lazy.Value    R:[Data_1]  35,23
57  T:34    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 36,3
60  T:12    M:GetData   E:end_get_lazy.Value    R:[Data_1]  36,52
72  T:35    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 37,29
63  T:33    M:GetData   E:end_get_lazy.Value    R:[Data_2]  40,92
74  T:34    M:GetData   E:end_get_lazy.Value    R:[Data_2]  41,08
76  T:32    M:GetData   E:end_get_lazy.Value    R:[Data_2]  41,12
61  T:36    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 41,31
69  T:37    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 43,2
62  T:36    M:GetData   E:end_get_lazy.Value    R:[Data_2]  43,6
66  T:15    M:GetData   E:end_get_lazy.Value    R:[Data_1]  45,93
67  T:38    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 46,3
68  T:38    M:GetData   E:end_get_lazy.Value    R:[Data_2]  46,44
75  T:37    M:GetData   E:end_get_lazy.Value    R:[Data_2]  47,19
71  T:24    M:GetData   E:end_get_lazy.Value    R:[Data_1]  47,71
73  T:35    M:GetData   E:end_get_lazy.Value    R:[Data_2]  48,57
77  T:32    M:GetData   E:begin_call_TryRemove  R:- 49,97
78  T:32    M:GetData   E:end_call_TryRemove    R:True  50,04
79  T:50    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:True  50,8
80  T:51    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 51,58
81  T:44    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 51,58
82  T:41    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 52,3
83  T:49    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 52,3
84  T:58    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 52,78
85  T:42    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 53,3
86  T:57    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 53,3
90  T:52    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 53,3
119 T:39    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 53,33
112 T:45    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 53,43
87  T:59    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 54,53
88  T:54    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 55,29
89  T:47    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 57,1
92  T:46    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 57,29
93  T:43    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 57,33
91  T:53    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 58,29
94  T:60    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 59,56
95  T:50    M:GetDataInternal   E:Return    R:[Data_3]  60,29
96  T:50    M:GetData   E:end_get_lazy.Value    R:[Data_3]  60,37
99  T:51    M:GetData   E:end_get_lazy.Value    R:[Data_3]  60,69
97  T:50    M:GetData   E:begin_call_TryRemove  R:- 60,88
117 T:44    M:GetData   E:end_get_lazy.Value    R:[Data_3]  61,24
98  T:50    M:GetData   E:end_call_TryRemove    R:True  61,26
107 T:49    M:GetData   E:end_get_lazy.Value    R:[Data_3]  61,41
108 T:58    M:GetData   E:end_get_lazy.Value    R:[Data_3]  61,48
109 T:53    M:GetData   E:end_get_lazy.Value    R:[Data_3]  61,48
100 T:54    M:GetData   E:end_get_lazy.Value    R:[Data_3]  61,73
111 T:46    M:GetData   E:end_get_lazy.Value    R:[Data_3]  61,81
102 T:43    M:GetData   E:end_get_lazy.Value    R:[Data_3]  62,15
110 T:47    M:GetData   E:end_get_lazy.Value    R:[Data_3]  62,6
101 T:57    M:GetData   E:end_get_lazy.Value    R:[Data_3]  63,55
104 T:41    M:GetData   E:end_get_lazy.Value    R:[Data_3]  63,62
103 T:61    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:True  64,3
106 T:60    M:GetData   E:end_get_lazy.Value    R:[Data_3]  64,63
105 T:42    M:GetData   E:end_get_lazy.Value    R:[Data_3]  65,05
116 T:52    M:GetData   E:end_get_lazy.Value    R:[Data_3]  66,18
121 T:59    M:GetData   E:end_get_lazy.Value    R:[Data_3]  66,56
114 T:56    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 67,47
122 T:55    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 69,18
118 T:48    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 71,37
115 T:40    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 71,63
113 T:45    M:GetData   E:end_get_lazy.Value    R:[Data_3]  72,02
123 T:61    M:GetDataInternal   E:Return    R:[Data_4]  74,29
120 T:39    M:GetData   E:end_get_lazy.Value    R:[Data_3]  77,3
124 T:56    M:GetData   E:end_get_lazy.Value    R:[Data_4]  79,26
130 T:61    M:GetData   E:end_get_lazy.Value    R:[Data_4]  79,27
149 T:63    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 79,3
131 T:48    M:GetData   E:end_get_lazy.Value    R:[Data_4]  79,32
140 T:40    M:GetData   E:end_get_lazy.Value    R:[Data_4]  79,5
125 T:55    M:GetData   E:end_get_lazy.Value    R:[Data_4]  79,78
126 T:65    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 80,76
127 T:65    M:GetData   E:end_get_lazy.Value    R:[Data_4]  80,93
128 T:66    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 81,32
129 T:66    M:GetData   E:end_get_lazy.Value    R:[Data_4]  81,66
132 T:62    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 82,32
134 T:61    M:GetData   E:begin_call_TryRemove  R:- 82,55
138 T:68    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 83,3
142 T:74    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 83,31
133 T:62    M:GetData   E:end_get_lazy.Value    R:[Data_4]  83,37
136 T:70    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 84,94
135 T:61    M:GetData   E:end_call_TryRemove    R:True  84,95
141 T:75    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:True  84,95
147 T:72    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 85,1
137 T:64    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 86
152 T:70    M:GetData   E:end_get_lazy.Value    R:[Data_4]  86
162 T:77    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 86,29
144 T:67    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 86,3
139 T:68    M:GetData   E:end_get_lazy.Value    R:[Data_4]  86,89
151 T:69    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 88,22
143 T:74    M:GetData   E:end_get_lazy.Value    R:[Data_4]  88,98
155 T:78    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 89,47
145 T:71    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 89,51
148 T:76    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 90,3
146 T:73    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 90,37
150 T:63    M:GetData   E:end_get_lazy.Value    R:[Data_4]  94,68
153 T:64    M:GetDataInternal   E:Return    R:[Data_5]  96,6
154 T:64    M:GetData   E:end_get_lazy.Value    R:[Data_5]  97
158 T:75    M:GetData   E:end_get_lazy.Value    R:[Data_5]  97,04
168 T:69    M:GetData   E:end_get_lazy.Value    R:[Data_5]  97,15
170 T:73    M:GetData   E:end_get_lazy.Value    R:[Data_5]  97,17
166 T:76    M:GetData   E:end_get_lazy.Value    R:[Data_5]  97,2
157 T:67    M:GetData   E:end_get_lazy.Value    R:[Data_5]  97,35
156 T:78    M:GetData   E:end_get_lazy.Value    R:[Data_5]  98,39
165 T:72    M:GetData   E:end_get_lazy.Value    R:[Data_5]  99,16
169 T:71    M:GetData   E:end_get_lazy.Value    R:[Data_5]  99,91
159 T:75    M:GetData   E:begin_call_TryRemove  R:- 100,13
160 T:75    M:GetData   E:end_call_TryRemove    R:True  100,66
161 T:79    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 101,06
164 T:79    M:GetData   E:end_get_lazy.Value    R:[Data_5]  101,29
163 T:77    M:GetData   E:end_get_lazy.Value    R:[Data_5]  101,84
167 T:80    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:True  102,43
171 T:92    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 105,32
172 T:93    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 106,97
173 T:82    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 106,97
175 T:91    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 106,97
176 T:83    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 106,97
174 T:95    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 107,18
178 T:89    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 107,35
179 T:94    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 108,56
177 T:88    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 108,58
180 T:90    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 110,32
181 T:84    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 111,31
183 T:97    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 111,52
182 T:86    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 111,8
186 T:81    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 112,6
184 T:85    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 113,3
185 T:96    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 113,42
187 T:87    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 115,35
188 T:80    M:GetDataInternal   E:Return    R:[Data_6]  117,87
189 T:80    M:GetData   E:end_get_lazy.Value    R:[Data_6]  122,92
192 T:92    M:GetData   E:end_get_lazy.Value    R:[Data_6]  122,93
195 T:83    M:GetData   E:end_get_lazy.Value    R:[Data_6]  122,93
207 T:96    M:GetData   E:end_get_lazy.Value    R:[Data_6]  122,98
193 T:97    M:GetData   E:end_get_lazy.Value    R:[Data_6]  123,28
190 T:80    M:GetData   E:begin_call_TryRemove  R:- 123,3
199 T:85    M:GetData   E:end_get_lazy.Value    R:[Data_6]  123,42
202 T:91    M:GetData   E:end_get_lazy.Value    R:[Data_6]  123,45
209 T:87    M:GetData   E:end_get_lazy.Value    R:[Data_6]  123,54
194 T:90    M:GetData   E:end_get_lazy.Value    R:[Data_6]  123,68
191 T:80    M:GetData   E:end_call_TryRemove    R:True  123,75
197 T:84    M:GetData   E:end_get_lazy.Value    R:[Data_6]  124,62
205 T:82    M:GetData   E:end_get_lazy.Value    R:[Data_6]  124,78
201 T:81    M:GetData   E:end_get_lazy.Value    R:[Data_6]  125,03
198 T:86    M:GetData   E:end_get_lazy.Value    R:[Data_6]  125,25
196 T:93    M:GetData   E:end_get_lazy.Value    R:[Data_6]  125,58
200 T:95    M:GetData   E:end_get_lazy.Value    R:[Data_6]  126,03
206 T:94    M:GetData   E:end_get_lazy.Value    R:[Data_6]  126,22
203 T:89    M:GetData   E:end_get_lazy.Value    R:[Data_6]  127,19
204 T:100   M:GetData   E:end_call_GetOrAdd()-&gt;added    R:True  127,58
211 T:88    M:GetData   E:end_get_lazy.Value    R:[Data_6]  129
208 T:102   M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 129,3
210 T:98    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 130,3
212 T:99    M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 131,31
213 T:101   M:GetData   E:end_call_GetOrAdd()-&gt;added    R:False 136,3
214 T:100   M:GetDataInternal   E:Return    R:[Data_7]  138,3
215 T:100   M:GetData   E:end_get_lazy.Value    R:[Data_7]  138,4
218 T:102   M:GetData   E:end_get_lazy.Value    R:[Data_7]  138,4
221 T:101   M:GetData   E:end_get_lazy.Value    R:[Data_7]  138,56
219 T:99    M:GetData   E:end_get_lazy.Value    R:[Data_7]  138,58
216 T:100   M:GetData   E:begin_call_TryRemove  R:- 138,73
220 T:98    M:GetData   E:end_get_lazy.Value    R:[Data_7]  138,98
217 T:100   M:GetData   E:end_call_TryRemove    R:True  139,04