I "watched" the development of the Deer Isle Koster on the Wooden Boat forum, it's a gorgeous boat and a wonderful camp-cruiser.
About balanced lug sail trim...low stretch halyards are your friend. If the downhaul on the front of the boom is not 2:1 already, make it so. 3:1 ....mmmmm. Maybe. 4:1 is probably too much. However, you want that leading edge nice 'n tight. The most common balanced lugsail oopsie looks like this.
See the big crease from the back of the boom up forward to the front of the yard? If you get that, try two things...1.) easing off on the boom downhaul and then topping up the yard harder and then...2.) tightening down on the boom downhaul.
you have a little bit of this going on already in the photo you posted. You can also try making a change so the boom rides an inch or two further forward, relative to the mast.
Wonderful boat!
1968 Selmer Series 9 B-flat and A clarinets
1962 Buesher "Aristocrat" tenor saxophone
Piper One Design 24, Hull #35; "Alpha"