Upload File to Space Digitalocean With Python
DigitalOcean Spaces is an S3-uniform object storage service with a built-in Content Commitment Network (CDN). Like Amazon S3, it provides a uncomplicated, cost-effective way to shop static files.
This tutorial shows how to configure Django to load and serve up static and user uploaded media files, public and individual, via DigitalOcean Spaces.
Prefer to use S3? Check out Storing Django Static and Media Files on Amazon S3.
- DigitalOcean Spaces
- Django Project
- Django Storages
- Static Files
- Public Media Files
- Individual Media Files
- Determination
DigitalOcean Spaces
DigitalOcean Spaces vs AWS S3?
As mentioned, DigitalOcean Spaces is an object storage service. When compared to S3, DigitalOcean Spaces:
- is easier to apply
- has a simpler and more predicable pricing model
- has much better documentation
That said, DigitalOcean Spaces is less flexible than S3 and has much less regions compared.
DigitalOcean Spaces is much easier to set upward and use than S3. Adding on a CDN requires a few button clicks from the UI. Plus, it'due south just as developer-friendly as S3. If you're already using DigitalOcean, you should definitely utilize it.
Getting Started
Showtime, y'all'll need to sign up for a DigitalOcean account (if you don't already accept ane), and then generate Spaces admission keys so you can access the DigitalOcean Spaces API.
Go alee and create the keys. Yous should take an access key ID and a hush-hush access key. With them, you lot can interact with the Spaces API with Boto3.
Case:
import boto3 # configure session and client session = boto3 . session . Session () client = session . customer ( 's3' , region_name = 'nyc3' , endpoint_url = 'https://nyc3.digitaloceanspaces.com' , aws_access_key_id = 'YOUR_ACCESS_KEY_ID' , aws_secret_access_key = 'YOUR_SECRET_ACCESS_KEY' , ) # create new bucket client . create_bucket ( Bucket = 'your-bucket-name' ) # upload file with open up ( 'exam.txt' , 'rb' ) as file_contents : client . put_object ( Bucket = 'your-bucket-proper noun' , Key = 'test.txt' , Body = file_contents , ) # download file client . download_file ( Bucket = 'your-bucket-name' , Fundamental = 'test.txt' , Filename = 'tmp/test.txt' , ) For more examples, cheque out Using DigitalOcean Spaces with AWS S3 SDKs.
Side by side, from the DigitalOcean control panel, click "Create" in the top right, and and so click "Spaces" in the dropdown. Select a region, enable a CDN, select "Restrict File List", and create a unique name for your saucepan. Then, create the saucepan.
You should encounter something like to:
Django Project
Clone down the base project from the django-digitalocean-spaces repo on GitHub:
$ git clone -b base https://github.com/testdrivenio/django-digitalocean-spaces $ cd django-digitalocean-spaces From the project root, create the images and spin up the Docker containers:
$ docker-etch upwards -d --build Once the build is complete, collect the static files:
$ docker-compose exec spider web python manage.py collectstatic And then, navigate to http://localhost:1337:
You should be able to upload an image, and and so view the image at http://localhost:1337/media/IMAGE_FILE_NAME.
The radio buttons, for public vs. private, exercise not piece of work. We'll be adding this functionality later in this tutorial. Ignore them for now.
Take a quick await at the projection structure before moving on:
├── .gitignore ├── LICENSE ├── README.md ├── app │ ├── Dockerfile │ ├── hello_django │ │ ├── __init__.py │ │ ├── asgi.py │ │ ├── settings.py │ │ ├── urls.py │ │ └── wsgi.py │ ├── manage.py │ ├── mediafiles │ ├── requirements.txt │ ├── static │ │ └── bulma.min.css │ ├── staticfiles │ └── upload │ ├── __init__.py │ ├── admin.py │ ├── apps.py │ ├── migrations │ │ └── __init__.py │ ├── models.py │ ├── templates │ │ └── upload.html │ ├── tests.py │ └── views.py ├── docker-compose.yml └── nginx ├── Dockerfile └── nginx.conf Desire to learn how to build this projection? Check out the Dockerizing Django with Postgres, Gunicorn, and Nginx blog post.
Django Storages
Side by side, permit's install django-storages, then that nosotros tin can use Spaces as the main Django storage backend, and boto3, in order to to interact with the Spaces API.
Update the requirements file:
boto3==i.18.36 Django==3.two django-storages==1.11.1 gunicorn==20.one.0 Add storages to the INSTALLED_APPS in settings.py:
INSTALLED_APPS = [ 'django.contrib.admin' , 'django.contrib.auth' , 'django.contrib.contenttypes' , 'django.contrib.sessions' , 'django.contrib.messages' , 'django.contrib.staticfiles' , 'upload' , 'storages' , ] Update the images and spin up the new containers:
$ docker-compose up -d --build Static Files
Moving along, we need to update the handling of static files in settings.py:
STATIC_URL = '/static/' STATIC_ROOT = BASE_DIR / 'staticfiles' STATICFILES_DIRS = ( BASE_DIR / 'static' ,) MEDIA_URL = '/media/' MEDIA_ROOT = BASE_DIR / 'mediafiles' Replace those settings with the post-obit:
USE_SPACES = bone . getenv ( 'USE_SPACES' ) == 'Truthful' if USE_SPACES : # settings AWS_ACCESS_KEY_ID = bone . getenv ( 'AWS_ACCESS_KEY_ID' ) AWS_SECRET_ACCESS_KEY = bone . getenv ( 'AWS_SECRET_ACCESS_KEY' ) AWS_STORAGE_BUCKET_NAME = os . getenv ( 'AWS_STORAGE_BUCKET_NAME' ) AWS_DEFAULT_ACL = 'public-read' AWS_S3_ENDPOINT_URL = 'https://nyc3.digitaloceanspaces.com' AWS_S3_OBJECT_PARAMETERS = { 'CacheControl' : 'max-age=86400' } # static settings AWS_LOCATION = 'static' STATIC_URL = f 'https:// { AWS_S3_ENDPOINT_URL } / { AWS_LOCATION } /' STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage' else : STATIC_URL = '/static/' STATIC_ROOT = BASE_DIR / 'staticfiles' STATICFILES_DIRS = ( BASE_DIR / 'static' ,) MEDIA_URL = '/media/' MEDIA_ROOT = BASE_DIR / 'mediafiles' Exist sure to update the
AWS_S3_ENDPOINT_URLif you're using a different region thanNYC3.
Have annotation of USE_SPACES and STATICFILES_STORAGE:
- The
USE_SPACESsurroundings variable is used to turn the Spaces storage on (value isTruthful) and off (value isImitation). So, you could configure ii Docker compose files: One for development with Spaces off and the other for production with Spaces on. - The
STATICFILES_STORAGEsetting configures Django to automatically add static files to the Spaces bucket when thecollectstaticcommand is run.
Review the official django-storages documentation on DigitalOcean and Amazon S3 for more info on the above settings and config.
Add together the appropriate environment variables to the web service in the docker-etch.yml file:
web : build : ./app command : bash -c 'while !</dev/tcp/db/5432; do slumber 1; washed; gunicorn hello_django.wsgi:application --bind 0.0.0.0:8000' volumes : - ./app/:/usr/src/app/ - static_volume:/usr/src/app/staticfiles - media_volume:/usr/src/app/mediafiles expose : - 8000 environment : - SECRET_KEY=please_change_me - SQL_ENGINE=django.db.backends.postgresql - SQL_DATABASE=postgres - SQL_USER=postgres - SQL_PASSWORD=postgres - SQL_HOST=db - SQL_PORT=5432 - DATABASE=postgres - USE_SPACES=TRUE - AWS_ACCESS_KEY_ID=UPDATE_ME - AWS_SECRET_ACCESS_KEY=UPDATE_ME - AWS_STORAGE_BUCKET_NAME=UPDATE_ME depends_on : - db Don't forget to update
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEYwith the user keys that you just created along with theAWS_STORAGE_BUCKET_NAME.
To test, re-build and run the containers:
$ docker-etch downwards -five $ docker-etch up -d --build Collect the static files:
$ docker-etch exec web python manage.py collectstatic It should take much longer than earlier since the files are being uploaded to the Spaces bucket.
http://localhost:1337 should even so render correctly:
View the folio source to ensure the CSS stylesheet is pulled in from the Spaces bucket:
Verify that the static files can be seen on the DigitalOcean control panel within the "static" subfolder of the Spaces bucket:
Media uploads will still hit the local filesystem since we've but configured Spaces for static files. We'll piece of work with media uploads soon.
Finally, update the value of USE_SPACES to FALSE and re-build the images to make sure that Django uses the local filesystem for static files. Once done, change USE_SPACES back to TRUE.
To prevent users from overwriting existing static files, media file uploads should exist placed in a different subfolder in the saucepan. We'll handle this past creating custom storage classes for each type of storage.
Add together a new file chosen storage_backends.py to the "app/hello_django" folder:
from django.conf import settings from storages.backends.s3boto3 import S3Boto3Storage class StaticStorage ( S3Boto3Storage ): location = 'static' default_acl = 'public-read' class PublicMediaStorage ( S3Boto3Storage ): location = 'media' default_acl = 'public-read' file_overwrite = False Make the following changes to settings.py:
USE_SPACES = bone . getenv ( 'USE_SPACES' ) == 'TRUE' if USE_SPACES : # settings AWS_ACCESS_KEY_ID = os . getenv ( 'AWS_ACCESS_KEY_ID' ) AWS_SECRET_ACCESS_KEY = bone . getenv ( 'AWS_SECRET_ACCESS_KEY' ) AWS_STORAGE_BUCKET_NAME = os . getenv ( 'AWS_STORAGE_BUCKET_NAME' ) AWS_DEFAULT_ACL = 'public-read' AWS_S3_ENDPOINT_URL = 'https://nyc3.digitaloceanspaces.com' AWS_S3_OBJECT_PARAMETERS = { 'CacheControl' : 'max-age=86400' } # static settings AWS_LOCATION = 'static' STATIC_URL = f 'https:// { AWS_S3_ENDPOINT_URL } / { AWS_LOCATION } /' STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage' # public media settings PUBLIC_MEDIA_LOCATION = 'media' MEDIA_URL = f 'https:// { AWS_S3_ENDPOINT_URL } / { PUBLIC_MEDIA_LOCATION } /' DEFAULT_FILE_STORAGE = 'hello_django.storage_backends.PublicMediaStorage' else : STATIC_URL = '/static/' STATIC_ROOT = BASE_DIR / 'staticfiles' MEDIA_URL = '/media/' MEDIA_ROOT = BASE_DIR / 'mediafiles' STATICFILES_DIRS = ( BASE_DIR / 'static' ,) With the DEFAULT_FILE_STORAGE setting now prepare, all FileFields will upload their content to the Spaces bucket. Review the remaining settings before moving on.
Next, let's make a few changes to the upload app.
app/upload/models.py:
from django.db import models class Upload ( models . Model ): uploaded_at = models . DateTimeField ( auto_now_add = True ) file = models . FileField () app/upload/views.py:
from django.conf import settings from django.cadre.files.storage import FileSystemStorage from django.shortcuts import render from .models import Upload def image_upload ( asking ): if request . method == 'POST' : image_file = request . FILES [ 'image_file' ] image_type = request . POST [ 'image_type' ] if settings . USE_SPACES : upload = Upload ( file = image_file ) upload . save () image_url = upload . file . url else : fs = FileSystemStorage () filename = fs . salvage ( image_file . proper name , image_file ) image_url = fs . url ( filename ) return render ( request , 'upload.html' , { 'image_url' : image_url }) return render ( asking , 'upload.html' ) Create the new migration file and then build the new images:
$ docker-compose exec web python manage.py makemigrations $ docker-compose down -v $ docker-compose upwards -d --build $ docker-compose exec web python manage.py drift Test it out! Upload an image at http://localhost:1337. The image should exist uploaded to Spaces (to the media subfolder) and the image_url should include the S3 url:
Add a new class to the storage_backends.py:
form PrivateMediaStorage ( S3Boto3Storage ): location = 'private' default_acl = 'individual' file_overwrite = Faux custom_domain = Fake Add the appropriate settings:
USE_SPACES = os . getenv ( 'USE_SPACES' ) == 'True' if USE_SPACES : # settings AWS_ACCESS_KEY_ID = bone . getenv ( 'AWS_ACCESS_KEY_ID' ) AWS_SECRET_ACCESS_KEY = bone . getenv ( 'AWS_SECRET_ACCESS_KEY' ) AWS_STORAGE_BUCKET_NAME = os . getenv ( 'AWS_STORAGE_BUCKET_NAME' ) AWS_DEFAULT_ACL = 'public-read' AWS_S3_ENDPOINT_URL = 'https://nyc3.digitaloceanspaces.com' AWS_S3_OBJECT_PARAMETERS = { 'CacheControl' : 'max-historic period=86400' } # static settings AWS_LOCATION = 'static' STATIC_URL = f 'https:// { AWS_S3_ENDPOINT_URL } / { AWS_LOCATION } /' STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage' # public media settings PUBLIC_MEDIA_LOCATION = 'media' MEDIA_URL = f 'https:// { AWS_S3_ENDPOINT_URL } / { PUBLIC_MEDIA_LOCATION } /' DEFAULT_FILE_STORAGE = 'hello_django.storage_backends.PublicMediaStorage' # individual media settings PRIVATE_MEDIA_LOCATION = 'private' PRIVATE_FILE_STORAGE = 'hello_django.storage_backends.PrivateMediaStorage' else : STATIC_URL = '/static/' STATIC_ROOT = BASE_DIR / 'staticfiles' MEDIA_URL = '/media/' MEDIA_ROOT = BASE_DIR / 'mediafiles' STATICFILES_DIRS = ( BASE_DIR / 'static' ,) Create a new model in app/upload/models.py:
from django.db import models from hello_django.storage_backends import PublicMediaStorage , PrivateMediaStorage class Upload ( models . Model ): uploaded_at = models . DateTimeField ( auto_now_add = True ) file = models . FileField ( storage = PublicMediaStorage ()) class UploadPrivate ( models . Model ): uploaded_at = models . DateTimeField ( auto_now_add = True ) file = models . FileField ( storage = PrivateMediaStorage ()) So, update the view:
from django.conf import settings from django.core.files.storage import FileSystemStorage from django.shortcuts import render from .models import Upload , UploadPrivate def image_upload ( asking ): if request . method == 'POST' : image_file = request . FILES [ 'image_file' ] image_type = asking . POST [ 'image_type' ] if settings . USE_SPACES : if image_type == 'private' : upload = UploadPrivate ( file = image_file ) else : upload = Upload ( file = image_file ) upload . save () image_url = upload . file . url else : fs = FileSystemStorage () filename = fs . save ( image_file . name , image_file ) image_url = fs . url ( filename ) return render ( request , 'upload.html' , { 'image_url' : image_url }) return render ( request , 'upload.html' ) Once again, create the migration file, re-build the images, and spin upwards the new containers:
$ docker-compose exec web python manage.py makemigrations $ docker-compose down -v $ docker-compose up -d --build $ docker-etch exec web python manage.py drift To test, upload a private prototype at http://localhost:1337. Like a public image, the paradigm should be uploaded to Spaces (to the private subfolder) and the image_url should include the Spaces URL along with the following query string parameters:
- AWSAccessKeyId
- Signature
- Expires
Substantially, we created a temporary, signed URL that users can access for a specific flow of time. You lot won't be able to access it directly, without the parameters.
Conclusion
This postal service walked you through how to create a saucepan on DigitalOcean Spaces and prepare Django to upload and serve static files and media uploads to and from Spaces.
By using Spaces, you:
- Increment the amount of space you lot accept available for static and media files
- Decrease the stress on your ain server since it no longer has to serve upwardly the files
- Tin can limit access to specific files
- Can take reward of a CDN
Let us know if we missed annihilation or if y'all have any other tips and tricks. You can find the final lawmaking in the django-digitalocean-spaces repo.
morseasereardscon.blogspot.com
Source: https://testdriven.io/blog/django-digitalocean-spaces/
0 Response to "Upload File to Space Digitalocean With Python"
Postar um comentário