The only tool in COPTR that supports this is Wayback. Some time ago, in order to make it easier to use, I created a command line tool that launches Wayback with a sensible default configuration, called wayback-player.
There is also a set of related Python tools, warcprox, pywb (Python Wayback), and pywb-webrecorder. The Python Wayback implementation is easy to install (pip install pywb) and just needs an appropriate configuration file in order to work.