I am having an issue with netflow resolving dns names. This started after upgrading netflow from v2 to v3. I am running Orion v8.5.1 w/ SP3 and Netflow v3. All my nodes in Orion show up in netflow with the correct name, reverse lookups work fine. About a month after the upgrade, all the names disappeared and netflow only showed IP addresses. Also, the nslookup button on the endpoint details screen would do nothing. I verified reverse lookups work fine on my orion server, as far as windows is concerned. Restarting netflow and rebooting the server had no affect.... So I opened a support ticket... thier answer:
"If you truncate the FlowCorrelation table within the NetperfMon database, all netflow endpoints should be resolved to their DNS names again."
I did this, and after wating 2 days for NTA to automatically lookup hostnames, everything was back to normal. I expressed concern with Solarwinds support this was not an acceptable fix. Truncating this table everytime netflow looses it mind, is not a fix. Well... it has been about a month, and gues what, netflow has stopped resolving dns names again.... I can fix this by truncating the table again, but would like to find a REAL solution. Anyone have this same problem, or have a fix for it?
Thanks,
Gary Sample
I am having the exact same issue. Did you ever find a solution?How do you "truncate the FlowCorrelation table within the NetperfMon database". Are you talking about just removing all the records from that table? Is that going to cause any data loss?
Hi Guys,
This is a known bug in 3.0 and we are addressing it. The flow correlation table will rebuild so no history lost.
Andy
So, that means it is safe to run "TRUNCATE TABLE FlowCorrelation"?
It is safe in that the FlowCorrelation table only keeps a resolution of DNS resolutions. Once you delete it, of course, the IP addresses will not have a way to resolve in your graphs, until new NetFlows come in with those IP addresses in them. Then the service will see that they are not in the FlowCorrelation table and will make new DNS queries. The history will be retained, but if no new NetFlows come in for a particular IP address, then since the graphs join on the FlowCorrelation table, I believe that they won't show up until a new DNS query is made on them. Let me come up with a better way to do this instead of truncating the table. I should have some T-SQL for you to run instead this evening that will do the same thing in effect, but doesn't have this type of side-effect.
yes, I am still having this issue as well. I either have to truncate the table, or re-applying the latest netflow service pack and rebooting seems to fix it temporarily.
Here is what I really suggest that you run on the NetPerfMon database instead of truncating the FlowCorrelation table..
DECLARE @CacheExpiration datetimeSELECT @CacheExpiration = dateadd(second, 1, LastCacheExpirationCheck) FROM NetFlowCorrelationStateUPDATE FlowCorrelation SET CacheExpiration = @CacheExpiration
This will cause all the DNS entries for all IP addresses to expire and the service then will start performing DNS queries against all of them. Keep in mind that this is not a trivial task that you are forcing the service to do. It WILL take some time to make a network called to resolve each of the IPs in the FlowCorrelation. You can check on the service's progress by periodically running the following T-SQL:
SELECT Count(*) FROM NetFlowAddressToResolve
It will give you a count of how many more DNS queries that the service needs to make before it is finished. Note: If you have a bunch of sources sending NetFlows at the same time, this process can take a long time.
Hope this helps! Let me know!
So the values must be messed in somewhere for these dates. Can you look in the FlowCorrelation table and look now at what the dates are for the CacheExpiration field? (All 7 million should be the same since we ran the query)
Furthermore if you open the NetFlowCorrelationState table, what value is in there?
What is the computer system date/time on both the Orion/NTA machine and the SQL database server? Are they in the same timezones?
The CacheExpiration value for all the records is "8/21/2008 11:54:38 AM". That was about the time I ran your query.The only value in the NetFlowCorrelationState is "8/21/2008 1:29:16 PM" (the current time on the server).The MSSQL service is running on the Orion server itself.
I'm sorry. I took the wrong approach here. But the best thing to do here. (Even if you hadn't run the above T-SQL which updated the CacheExpiration dates on all those rows), is to run the following query...
INSERT INTO NetFlowAddressToResolve (IPAddress, Reason, Timestamp) SELECT IPAddressSort, 0, GETDATE()
FROM FlowCorrelation WITH(NOLOCK)
Again you can check its progress with
Looks like that worked! I ran the INSERT INTO command and I got (7122945 row(s) affected). And when I check the NetFlowAddressToResolve table, it started at about 7 million and is going down slowly. The domain names are starting to show up in the NetFlow pages now. This is going to be fixed in a NetFlow v3 service pack, right?
Any real fix for this issue yet? Looks like my netflow just lost its mind again... no DNS names are resolving.... I am running netflow SP4... support had me install it last week to fix a report issue.
For now it looks like i get to truncate the table again!
I'm seeing the same problem. Curious as to when the fix will be out...Thx
I have fought with this for a long time and posted an answer for this in another topic... It is not a complete solution, but it is a valid workaround and requires no server-side scripting. Hope it works for you!