Caulk/Putty around Interior Timber Frame Window Gaps
Yes, that appears to be old, dried caulk. The new caulks don't crack like they used to. Get yourself a tube of silicone caulk. It comes in clear and white and possibility other colors. Make sure you get one that's paintable. Clean out the old gap completely. Spraying in some white vinegar will help dissolve the old caulk and make it easier to remove. Wipe down with water and let dry completely. then apply new caulk, following the directions on the tube.