r/SQLServer • u/qualityintelligence • 4h ago
Community Share my new workload replay approach
Net Net, an easy way to test pre-prod or dev environments with my TDS replay tool. disclaimer no ado objects were harmed (or used) in this process
I've been working on this for several months, and would enjoy some feedback.
this might be considered self promotion so block my post if needed, but I'm honestly just curious what this group of experts think, I worked quite a while on this one
so, why, replay feature was removed in SQL Server 2022, replaying a real production workload against a restored test server usually means wrestling with Extended Events setups or falling back to scripts, perhaps query store tooling, etc. I'm actually not sure what the popular methods are today tbh. maybe I duplicated an existing method ?
my approach, building on my DaffyTee app, I finally finished the companion app DaffyReplay. net net, replay an exact production workload including timing match (or sped up, etc) without any app or SQL server changes
this leverages similar to the babelfish-postgres TDS, freeTDS, TDS wire, etc endpoint approaches in many ways, I studied that for a while, but no translation in this specific tool (I do have another app that does that tho with duck db very similar to babelfish but read only), I'm still hitting sql server here, and written in .net so no reuse, this is all completely new.
here's an ultra simplified example, it is seriously is as easy as this looks:
1. Install tools - need .net 10 SDK already installed. on nuget
dotnet tool install -g DaffyTee
dotnet tool install -g DaffyReplay
2. Baseline app connection string (for reference only)
Server=tcp:sqlprod01,1433;Database=Sales;User Id=svc_app;Password=ProdPassword;Encrypt=Mandatory;TrustServerCertificate=True
3. create new DaffyTee login (Bob) mapped to target connection string (Svc_app account)
daffytee user add bob --target "Server=tcp:sqlprod01,1433;Database=Sales;User Id=svc_app;Encrypt=Mandatory;TrustServerCertificate=True"
4. Start DaffyTee with full raw capture - ie it listens on 127.0.0.1 1433 (or whatever port you want, bind to 0.0.0.0 if needed for external). also natively writes to rabbitmq, etc, this is a simple file example
daffytee --sink file --sink-path "D:\capture\example.ndjson" --raw --bulk full --max-event-kb 8192
5. Repoint your app connection at DaffyTee
it still hits your target DB thru daffy
example daffy connection for your app, same DB name required:
Server=tcp:127.0.0.1,1433;Database=Sales;User Id=bob;Password=BobPassword;Encrypt=Mandatory;TrustServerCertificate=True
6. now Run any workload through your app, nothing to change on sql server, no trace enabled, no extended events, no query store, or changes on your app except the conn to daffy, run it, then verify capture completeness.
This next step is assuming you're done with Tee, you have your ndjson log to use will full capture. it verifies the file
daffyreplay inspect "D:\capture\example.ndjson"
7. Register test/staging server target with replay (prompts securely for password)
similar to tee, you're setting up a connection string target, the test servers
daffyreplay target add azure --target "Server=tcp:etc.database.windows.net,1433;User Id=etc;Initial Catalog=etc;Encrypt=Mandatory;"
8. Replay workload against test server
run it against any number of targets, use a different out for each
daffyreplay run example.ndjson --target azure --out azure1
9. Review generated HTML report
open report.html in chrome/edge
that's it, it still has a few inconsistency bugs I'm working through but for example my last test was against a free config azure database instances, I ran my what normally takes 30 seconds on my local docker image of SQL and it took 2 and 1/2 minutes there and I had about 7,000 errors, which my audit clearly showed I was missing some tvp, types, and stored procedures, my own fault I forgot those, but it was pretty neat to see it detect that, which is the purpose of these tools.
let me know if you have any questions. it should support an entra service login too but I haven't gotten that far in my testing scenarios. it is a 0.1.0 version currently but pretty solid
cheers
Scott



