Hi, guys
I'm trying to figure out a way of updating a lot of records, i'm building an application that polls network devices every minute, and stores the relevant data to a database. In doing so i have made a "RRD" (Round Robin Database) (every network device has 1440 records in a table. The 1440 records represent every minute of a day.)
The problem is that I have up to several thousand different network devices to poll. The polling is no problem, but the updating of the records in the database poses a problem as it takes way to long. Please if anyone have any ideas on how to deal with this, either another way of storing the data, or an efficient way of updating the records in the database.
I’m using Visual basic 2005 and SQL 7.0.
Thanks,
Martin
Martin RosjordePosted Jun 11, 2007, 4:41 AM
I have tried the updatebatchsize, but the problem maybe in the way i try to update the database. Do you have any other way of doing this?
Cheers,
Martin
Dim connect As New SqlConnection(ConnectionString)
connect.Open()
Dim command As String = "SELECT * from tblpingRTT"
Dim DS As New DataSet()
Dim adaptor As SqlDataAdapter = New SqlDataAdapter(command, connect)
adaptor.UpdateBatchSize = 0
Dim cmdBuilder As SqlCommandBuilder = New SqlCommandBuilder(adaptor)adaptor.MissingSchemaAction = MissingSchemaAction.AddWithKeyadaptor.Fill(DS, "tblPingRTT")
Dim dt As DataTable = DS.Tables("tblpingrtt") Dim en As IEnumerator = UpdatePingRTT.GetEnumerator While en.MoveNext
Dim RTT As New PingRTT
RTT = en.Current Dim sql As String = "DeviceID=" & RTT.DeviceID & " and timeinterval=" & GetTimer()
Dim mydatarow = dt.Select(sql)(0)
mydatarow("TimeInterval") = GetTimer()
mydatarow("Value") = RTT.RTT
mydatarow("DeviceID") = RTT.DeviceID End While
UpdatePingRTT.Clear()
adaptor.UpdateCommand = cmdBuilder.GetUpdateCommand()
adaptor.Update(DS.Tables(0))Mike GoldPosted Jun 8, 2007, 5:09 PM
Check this tip for SQL Server
In ADO.NET 2.0 You can also use the DataAdapter and set an UpdateBatchSize to set the number of commands that will be sent to the database with each request to optimize your update call.
In your adapter:
Execute the update process:
adapter.Update(ds);