How long did you let the fresco dry before you begun secco? I would think that you did not wait long enough,
I would leave it for six month before coming with touchups if ever (do not believe in secco so to speak).

If the wall is not dry enough than that acrylic that you mix into your tempera creates a film which stops fresco from "breathing" - hense the flaking.

The eggtempera would be better choice - eggs are often added to plaster - they will not clogg it and think it would be better binder for a fresh wall secco.

PS. Post some pics - this is an impressive size fresco.