TaskFactory WebRequests and DataBase Writing

[mickey_pt]

Need help or at least check what I'm doing wrong.

I have a windows service that runs everyday at a schedule time, it takes a list of internal IDs and a related URL, after dividing the list into batches (i'm using batches with 2 elemnts), the code loop through all batches, and make web requests and write the result to the database (log) for each id in the current batch. Each request has is own thread and a predefined/static timeout.

I'm having two problems:

First one, the log, are written but the first time it runs it kind duplicate the first Id that it founds in the list, sample at the bottom.

Second one, this one, i don't know if it's "my" problem, or the website, it's always the same URL, only different QS parameteres. I have a static timeout for each request, in the sample 3 seconds, after the second request I always get timeout.

List<List<KeyValuePair<long, string>>> batchList = new List<List<KeyValuePair<long, string>>>(10);
 
//Get Credentials and timeout value one time
ICredentials credentials = GetCredentials(); //Get default credentials, doesnt change
int clickTimeout = 3; //will be seconds
int j = 1;
 
//Loop
foreach (List<KeyValuePair<long, string>> batch in batchList)
{
    LogHelper.InsertJobLog("ServiceName", string.Format("Executing batch: {0}.", (j).ToString()), reference);
    HandleProcessBatch(batch, credentials, clickTimeout);
    LogHelper.InsertJobLog("ServiceName", string.Format("End Executing batch: {0}.", (j).ToString()), reference);
    j++;
}
 
// ....
 
//Handle batches of processes
private static void HandleProcessBatch(List<KeyValuePair<long, string>> procList, ICredentials credentials, int clickTimeout)
{
    try
    {
        //Task list
        List<Task> listaTarefas = new List<Task>();
        foreach (KeyValuePair<long, string> proc in procList)
        {
            LogHelper.InsertJobLog("ServiceName", string.Format("ProcID {0} will be added to the execution list: {1}", proc.Key, proc.Value), null);
            listaTarefas.Add(
                Task.Factory.StartNew(
                    () => SimulateURLRequest(proc.Key,proc.Value, credentials, clickTimeout)
                    )
                );
        }
 
        //Wait for all tasks in this list to end
        Task.WaitAll(listaTarefas.ToArray());
    }
    catch (Exception ex)
    {
        EventLogWrite.Error("ServiceName-HandleProcessBatch", ex.Message);
    }
 
}
 
//Called in diferrent threads
private static void SimulateURLRequest(long processID, string processURL, ICredentials credentials, int timeout)
{
    try
    {
        Uri ipacURL;
        if (!Uri.TryCreate(processURL, UriKind.Absolute, out ipacURL) || (ipacURL.Scheme != Uri.UriSchemeHttp && ipacURL.Scheme != Uri.UriSchemeHttps))
        {
            LogHelper.InsertJobLog("ServiceName", string.Format("Proc {0} URL not valid: {1}", processID.ToString(), processURL),null);
            return;
        }
        WebRequest wr = WebRequest.Create(ipacURL); 
        wr.Credentials = credentials;
        wr.Timeout = (timeout * 1000);
        HttpWebResponse answer = (HttpWebResponse)wr.GetResponse();
 
        LogHelper.InsertJobLog("ServiceName", string.Format("Proc {0} result: {1}", processID.ToString(), Enum.GetName(typeof(HttpStatusCode), answer.StatusCode)),null);
    }
    catch (WebException wex)
    {
        LogHelper.InsertJobLog("ServiceName", wex.Message,null);           
    }
    catch (Exception ex)
    {
        LogHelper.InsertJobLog("ServiceName", ex.Message, null);
        EventLogWrite.Error("ServiceName-SimulateURLRequest", ex.Message);
    }
}

Logs sample:

ServiceName 2019-11-11 20:01:03.000 Daily Task Ended.
ServiceName 2019-11-11 20:01:02.000 End Executing batch: 2.
ServiceName 2019-11-11 20:01:02.000 The operation has timed out
ServiceName 2019-11-11 20:00:59.000 ProcID 20973800 will be added the the execution list: URL_20973800
ServiceName 2019-11-11 20:00:59.000 Executing batch: 2.
ServiceName 2019-11-11 20:00:59.000 End Executing batch: 1.
ServiceName 2019-11-11 20:00:59.000 Proc 20973792 result: OK
ServiceName 2019-11-11 20:00:59.000 Proc 20973792 result: OK
ServiceName 2019-11-11 20:00:58.000 ProcID 20973792 will be added the the execution list: URL_20973792
ServiceName 2019-11-11 20:00:58.000 ProcID 20972361 will be added the the execution list: URL_20972361
ServiceName 2019-11-11 20:00:58.000 Executing batch: 1.
ServiceName 2019-11-11 20:00:34.000 Daily Task Started.

The ID 20973792 was written two times to the database, and after that it starts to yield timeouts...

I don't see what I'm doing wrong...