The dog was hit when a man opened fire on his handler, Officer Justin Darby.
Darby was not injured in the shooting.
“This is a difficult time for not only Officer Darby and his family, but for our entire department. Sjaak, and the other K9s, are a part of our family," said Interim Police Chief Chip Davis. “A K9 is more than just a dog to our handlers, they’re a partner.”
Initial reports from the scene indicate that a motorist pulled alongside a police officer's vehicle after pulling out of the police department parking lot and shots were fired into the officer's car, an official with the Tennessee Bureau of Investigation told reporters.
Police then pursued the motorist into the parking lot of an apartment complex where a shootout ensued.
The suspect was taken to a hospital with a gunshot wound, and later died.
It wasn't until Darby got back to his car after the shootout that he realized his dog had been hit.
Sjaak is the first line of duty death for the La Vergne Police Department. The agency said the K9 will be buried with full honors.
FOX News contributed to this report.