AMC: "too many workflows have already been run against the same input data sources"
AMC rejects an execution when too many similar queries already ran on the same data sources and time windows.
Message from Amazon
Execution "X" of workflow "Y" could not be run because too many workflows have already been run against the same input data sources in the same or overlapping time windows.Symptom
The workflow execution does not start and is rejected with the message quoted above.
Cause
Before each execution AMC checks the query against earlier executions. Running several similar queries with slightly different filters on non-sensitive fields could be used to derive information about a sensitive value. AMC blocks this to protect the identity of the audience information in the datasets.
Fix
- Wait and run the query again once the overlapping executions are further back.
- Combine queries instead of running many variants of the same filter one after another.
- Choose the time windows and data sources of the variants differently where the analysis allows it.
Practical tip (our own take): Instead of one query per product with WHERE asin = ..., group by the dimension once and filter the result afterwards.
The cause follows the Amazon documentation. Some of the fix steps are our own judgement and do not replace asking Amazon support.