we have custom logging system. single dll pushes logs microsoft sql server. logs viewed via webpage, advanced sorting/filtering/paging/download. built system scratch , new entity framework , not super strong in sql.
one co-worker insisting windows event log better choice sql server, save on bandwidth, therefore being quicker. tried , true solution.
i questioning practicality of using windows event log handle our extensive logging. in sql have single log table hosts around 150 million logs received ~100 different machines across ~600 applications 90gb total.
is using windows event log viable solution kind of load? creating way admin server/user request/sort/filter/page logs without intermediary sql? or blinded due having built current system?
additional info
log single table log(id, appid, type, lvl, source, msg, time)
150 million logs covers errors 7 days, else 3 days
normalizing source , msg drastically reduce size of table there many duplicate messages.
most complaints current system webpage times out on applications when requesting logs. timeout still 30 seconds, logs limited 10,000 total, @ point have refine search. timeout occurs on applications have average of 800k+ logs. other apps return results @ average max of 10 seconds.
i making changes indexes , have thought normalizing table increase efficiency ongoing process. if windows event log better solution rather spend time implementing that.
this sole own opinion
windows event logging distributed systems bad idea.
your idea of centralizing logs applications great. statistics, have backup mechanism, can use sql server clustering, etc.
if people complain performance issues, maybe time optimizing solution. indices maybe? @ indexed. make sure sql server can make use of search arguments (sargs) when using conditions.
can split data multiple tables? maybe normalizing data should looked into.
you pre-made filters users use , make them available , work on them (performance)
this start. there's details don't know , can't disclose
Comments
Post a Comment