5 more episodes yet so I'd be surprised if this was the last we'd see of Chase, though after Season's 4 ending, I guess you never know.
I agree with your theory that he probably has something bigger planned but he also likes getting into Oliver's head and playing mind games with him so I think he'd take any chance he can get to do that.
Not my favourite episode either but I did love the 'AND I LIKED IT' scene and the Russia stuff.
Thanks for your great reaction.