No, AFAIK adding a new architecture to a framework binary shouldn't change the notarization process at all.
First, let's verify that you don't get any "issues" in your notarization log. Run
xcrun altool --notarization-info $NOTARIZATION_ID_REPORTED_BY_ALTOOL -u $USERNAME -p $APP_SPECIFIC_PASSWORD
and look for the
LogFileURL field in the output. There should be zero "issues" in the log. Also note that there should be twice as many items as usual in "ticketContents" since every single binary contains 2 architectures now.
Second, let's verify that you're not actually mangling the framework bundle during the upload/download process. Take a copy of the plugin you've downloaded from the internet and verify the code signature:
spctl -a -t open --context context:primary-signature -v Foo.sketchplugin/Contents/Sketch/Foo.framework
(use whatever a subpath to the .framework bundle is in your case). This shoud give you something like
source=Notarized Developer ID
If spctl outputs anything else, use codesign to reveal a more detailed description of a problem:
codesign -vv Foo.sketchplugin/Contents/Sketch/Foo.framework
Btw, instead of uploading/downloading the plugin to make it "quarantined" by the OS (and thus a subject to security checks), you can add a quarantine flag yourself:
xattr -w com.apple.quarantine 1 Foo.sketchplugin/Contents/Sketch/Foo.framework