Media Sync Issue / Error 413:

Gramps Web
1

@DavidMStraub

So it seems that I’m getting regular Error 413 message when trying to sync files that I shouldn’t think should raise that error. I am unable to sync even 2MB image files. This is a two-tree multi-tree setup with MEDIA_PREFIX_TREE: True and “small” files syncing OK to the /app/media/{tree_ID} directory.

This is my nginx_proxy.conf

server {
    listen 443 ssl;  # Listen on port 443 for HTTPS traffic
    server_name #####-#####.org;  # Your domain

    # SSL certificates managed by acme-companion
    ssl_certificate /etc/nginx/certs/#####-#####.org/fullchain.pem;
    ssl_certificate_key /etc/nginx/certs/#####-#####.org/privkey.pem;

    client_max_body_size 500m;  # Set the maximum body size for this server block

    location / {
        proxy_pass http://grampsweb:5000;  # Forward all traffic to your Gramps Web service running on port 5000
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

nginx.1 | 2024/10/05 03:24:34 [error] 53#53: *72 client intended to send too large chunked body: 1048576+8192 bytes, client: 172.18.0.1, server: #####-########, request: "PUT /api/media/fb78ae61ee74d340aedcee224b4/file?uploadmissing=1 HTTP/1.1", host: "#####-########"

Would you concur that the nginx_proxy.conf is probably the issue?

OSX 14
Gramps 5.2.2

3

Yes see here the comment about nginx.conf
https://www.grampsweb.org/install_setup/lets_encrypt/

4

You should try to set

client_max_body_size 500m;

in both server and http sections.

1 Like
5

Thanks Gents.

@jmichault Thx. I tried that in the first 15 minutes after some google searches. No joy. (…and over the subsequent 4 hours, tried about 80 other things at the behest of chatGPT, stack-exchange and numerous articles on nginx.)

@DavidMStraub Yes. All that is configured correctly.

Nginx is a bit over my head, something tells me this is going to be a long and frustrating journey. I think the contents of my nginx conf file was given to me by chatGPT, I certainly didn’t figure it out, but in doing so, am clearly flying blind by my own hand.

Me and chatGPT will stick at it.

Thank you!

6

@DavidMStraub In re-reading your link, you use the words “the same directory”, but never say the “same directory as WHAT”? I assume it was the same directory as the docker-compose.yml file, so that’s where it is. Should it be in the same directory “as something else” you didn’t mention?

7

nevermind, ChatGPT gave me the answer. I’m one step smarter on nginx, but still a long way to go. The 413 error persists.

8

Have you reviewed:

9

Thx @emyoulation , I did, but that thread discussed an Apache config, not Docker setup, so it didn’t apply, or at least I interpreted it that way. If there was something to glean from it, I may have missed it.

I’m going to go at it again with fresh eyes today. Each ‘pass’ I learn a little more, so we’ll see where this goes. I’m only now getting comfortable poking around the Docker containers on the command line…so this will enable me to dig a little deeper.

10

What stood out to me was the question of whether you have an Ad Blocker installed and, if so, whether the 413 error persists without that tool active?

11

I’ll poke around that idea Brian…I do find that a bit curious, being that Ad-blockers tend to run on the browser (or so I think) and this is a sync issue at a “seemingly” a lower level. I mean, I can sync whether I’m running a browser or not if the server is up (and the file is small)…but clearly something odd is going on. Even chatGPT is out of ideas :stuck_out_tongue: so I’ll look into that.

I’ve learned a fair amount more about nginx and how the ‘flow’ of everything works and this is my latest nginx_proxy.conf and it seems to be quite reasonable. The investigation continues:

server {
    listen 80;  
    server_name family1-family2.org;
    chunked_transfer_encoding on;	
    client_max_body_size 100M;

    location / {
        return 301 https://$host$request_uri;  # Redirect to HTTPS
    }
}

server {
    listen 443 ssl;
    server_name family1-family2.org;
    chunked_transfer_encoding on;
    ssl_certificate /etc/nginx/certs/family1-family2.org.crt;
    ssl_certificate_key /etc/nginx/certs/family1-family2.org.key;
    client_max_body_size 100M;

    location / {
        proxy_pass http://grampsweb:5000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_buffering off;  # This disables buffering, sending data immediately to the backend
        proxy_buffers 16 16k;
        proxy_buffer_size 16k;
    }
}
1 Like
12

In the meantime, this is the nginx log: perhaps @DavidMStraub can glean something from it:

nginx.1     | family1-family2.org 172.19.0.1 - - [05/Oct/2024:16:14:31 +0000] "POST /token/ HTTP/1.1" 200 1377 "-" "Python-urllib/3.11" "172.19.0.3:5000"
nginx.1     | family1-family2.org 172.19.0.1 - - [05/Oct/2024:16:14:31 +0000] "POST /api/token/ HTTP/1.1" 200 1170 "-" "Python-urllib/3.11" "172.19.0.3:5000"
nginx.1     | family1-family2.org 172.19.0.1 - - [05/Oct/2024:16:14:31 +0000] "GET /api/exporters/gramps/file HTTP/1.1" 200 29181 "-" "Python-urllib/3.11" "172.19.0.3:5000"
nginx.1     | family1-family2.org 172.19.0.1 - - [05/Oct/2024:16:14:33 +0000] "GET /api/media/?filemissing=1 HTTP/1.1" 200 469 "-" "Python-urllib/3.11" "172.19.0.3:5000"
nginx.1     | 2024/10/05 16:14:34 [error] 34#34: *79 client intended to send too large chunked body: 1048576+8192 bytes, client: 172.19.0.1, server:  family1-family2.org, request: "PUT /api/media/fb78bef9b2dec4a809d32e71e9/file?uploadmissing=1 HTTP/1.1", host: " family1-family2.org"
nginx.1     | family1-family2org 172.19.0.1 - - [05/Oct/2024:16:14:35 +0000] "PUT /api/media/fb78bef9b2dec4a809d32e71e9/file?uploadmissing=1 HTTP/1.1" 413 183 "-" "Python-urllib/3.11" "-"
1 Like
13

Ok, I figured it out…taking “nginx 201” before “nginx 101” came with some pain for sure…about 8 hours total.

Due to my newness to Docker command line and ignorance of nginx, I was unawares of the ‘nginx.conf’ file for a time. I was focused on the ‘nginx_proxy.conf’ file and was putting the ‘client_max_body_size’ property in the ‘server’ section.

Now folks said to “put it in http”, but because I didn’t know about the nginx.conf file or much about that file contents in general, I was adding an ‘http’ section to the nginx_proxy.conf file, which broke the server.

It was only after quite a bit of chatGPT, poking around the Container that I started to see the structure and understand the ‘flow’ of things and eventually found my way to the ‘nginx.conf’ file where I then saw the HTTP section…and of course put the directive there and all was well.

Thanks for your patience folks…this will bear fruit in the form of more loquacious documentation as I now know how to address ‘dummies’ like me and can perhaps lower the barrier of entry to Gramps Web so more folks can enjoy it.

14

…so going full circle, and examining the nginx.conf file, I realize now that nginx_proxy.conf, being ‘included’ by nginx.conf, only needed the one line with ‘client_max_body_size’, just as @DavidMStraub has directed, but having had messed up other things with my fat fingers during my learning process, I did not make that connection and went down a longer learning road way out of the way. I had to go full circle to “understand it all”.

1 Like