[gimp/gimp-2-10] Issue #3893 - Error message when exporting to to TIF if there is a layer mask on a group

[Ell <ell src gnome org>]

commit 0c85a0b3e4a6448d7e1a94e0097e24bfb460c0c8
Author: Ell <ell_se yahoo com>
Date:   Sat Sep 7 10:24:51 2019 +0300

    Issue #3893 - Error message when exporting to to TIF if there is a layer mask on a group
    
    When exporting to a format that supports layers, but doesn't
    support layer masks, merge layer groups with a mask before applying
    their mask, since masks can't be applied to layer groups.
    
    (cherry picked from commit 643279bd319d80188ee7595a1e68e9dad3c30b61)

 libgimp/gimpexport.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
---
diff --git a/libgimp/gimpexport.c b/libgimp/gimpexport.c
index f0f5c1df15..b51b01a7e5 100644
--- a/libgimp/gimpexport.c
+++ b/libgimp/gimpexport.c
@@ -183,7 +183,15 @@ export_apply_masks (gint32  image_ID,
   for (i = 0; i < n_layers; i++)
     {
       if (gimp_layer_get_mask (layers[i]) != -1)
-        gimp_layer_remove_mask (layers[i], GIMP_MASK_APPLY);
+        {
+          /* we can't apply the mask directly to a layer group, so merge it
+           * first
+           */
+          if (gimp_item_is_group (layers[i]))
+            layers[i] = gimp_image_merge_layer_group (image_ID, layers[i]);
+
+          gimp_layer_remove_mask (layers[i], GIMP_MASK_APPLY);
+        }
     }
 
   g_free (layers);