Michael Handschuh
09cc78599a
We restrict each algorithm time loop to a pre-determined amount of time. Exceeding this limit will cause the algorithm to immediately terminate. This quickly becomes an issue when considering users running trainable models that have a long initialization period that exceeds the time loop maximum. This change provides a mechanism through which a long-running scheduled event is permitted to keep running and is permitted to avoid the time loop permitted by requesting additional time. Requests for additional time are limited according to a leaky bucket implementation whose parameters are set via the job's controls structure. The fundamental time unit for the algorithm is a single minute. Here's how it works. If a scheduled event takes longer than one full wall clock second then a request is made to the leaky bucket for one more minute. If the scheduled event continues to take more time, it will continue to request additional minutes. Each requested minute will prevent the algorithm's time loop check from terminating the algorithm. When the bucket is empty and no more minutes are available to be requested, a TimeoutException is thrown causing a cascade that ends in the algorithm's termination and status being flipped to RuntimeError. Additionally, this applies equally to ALL scheduled events. While some helpers were added with the naming of Train and TrainNow to the ScheduleManager, these methods don't do anything special and the infrastructure doesn't otherwise flag them as different, so this feature becomes part of the core Scheduled Event feature set. Further, the live scheduled events were not touched and are still pending further discussion regarding the value added by enforcing a time restriction when simulation time and wall clock time are equivalent. Fixes #3319
51 lines
2.0 KiB
C#
51 lines
2.0 KiB
C#
/*
|
|
* QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
|
|
* Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
* you may not use this file except in compliance with the License.
|
|
* You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software
|
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
* See the License for the specific language governing permissions and
|
|
* limitations under the License.
|
|
*/
|
|
|
|
using System;
|
|
using System.Threading;
|
|
|
|
namespace QuantConnect.Util.RateLimit
|
|
{
|
|
/// <summary>
|
|
/// Provides extension methods for interacting with <see cref="ITokenBucket"/> instances as well
|
|
/// as access to the <see cref="NullTokenBucket"/> via <see cref="TokenBucket.Null"/>
|
|
/// </summary>
|
|
public static class TokenBucket
|
|
{
|
|
/// <summary>
|
|
/// Gets an <see cref="ITokenBucket"/> that always permits consumption
|
|
/// </summary>
|
|
public static ITokenBucket Null = new NullTokenBucket();
|
|
|
|
/// <summary>
|
|
/// Provides an overload of <see cref="ITokenBucket.Consume"/> that accepts a <see cref="TimeSpan"/> timeout
|
|
/// </summary>
|
|
public static void Consume(this ITokenBucket bucket, long tokens, TimeSpan timeout)
|
|
{
|
|
bucket.Consume(tokens, (long) timeout.TotalMilliseconds);
|
|
}
|
|
|
|
/// <summary>
|
|
/// Provides an implementation of <see cref="ITokenBucket"/> that does not enforce rate limiting
|
|
/// </summary>
|
|
private class NullTokenBucket : ITokenBucket
|
|
{
|
|
public long Capacity => long.MaxValue;
|
|
public long AvailableTokens => long.MaxValue;
|
|
public bool TryConsume(long tokens) { return true; }
|
|
public void Consume(long tokens, long timeout = Timeout.Infinite) { }
|
|
}
|
|
}
|
|
} |