I can use anything, my last idea was adding this post-processed data in a Dictionary using chunks of 1000 rows then trying to do something like a Bulk insert then reading another 1000 and so on, i can use any container, but is there any way to do something like a bulk insert so i don't have to insert 1 record each time? I've been reading some examples about SqlBulkInsert but all these examples take data from a CSV file, maybe there is a way to do the same using some container in memory, like array/list/dictionary or a datatable.

I've come to a working version of whta i wanted, but it's ugly:

- I add 100 1000 records( taken from file and post processed) to a Dictionary ensuring they are all unique records (this is part of this post-processing)
- I use a StringBuilder to concatenate all queries like:

Insert into temp_table
select field1, field2, field3
union all
select field1, field2, field3
union all .. and so on 1000 times.

- After that in a stored procedure i join these temp table with the real table to the the records not present in the real table and just insert those records
- Finally i erase the temp table, erase the stringbuilder, and proceed to the next loop.

It's working fast now, but it's ugly, it's inserting a giant sql string, and also i dont like to create Sql Strings by concatenating them with values, it's not a good practice, i usually use SqlParameters when i can, but i can't use them when doing this ( i would need like 3K params i think)

So, if something like a Bulk-Insert can be made directly from data in memory, it would be the best.