- Compatible XF Versions
- 2.2
- Additional Requirements
- OpenStack Swift container or S3 bucket
Store your attachments in the cloud efficiently. Using this addon you can save bandwidth and storage space by separating your heavy attachments from the rest of the forum.
Yeah, seriously, you can pay as little as $1 a month to store 100 GB of attachments that your users will happily upload as soon as you let them.
Under the hood, this addon allows you to mount internal_data/ directory (or "filesystem", since it's powered by the flysystem) on cloud storage. There is an official guide that shows how to do it the hard way, but you might as well install this addon instead. Below you'll find a more detailed comparison between those two methods.
This addon does NOT change, in any way, the other "filesystem" called data/. It usually holds image attachment thumbnails, avatars and videos. There may or may not be a subsequent version that provides some capabilities around that space.
I am releasing this addon for free, in hopes that it will be useful to you, without warranty or support of any kind.
Features
Comparison with mounting a Flysystem adapter manually
Excessive byte shoveling
Problem: Simply mounting a Flysystem adapter as internal_data/ causes every file operation to behave as if the cloud storage was a local filesystem. That means that every download is streamed from cloud storage, through your server to the end user. You would likely be paying TWICE for the storage and nullifying the effect of any CDN.
Solution: This addon attempts to redirect the user to the public or private (recommended) URL to fetch the attachment, so that they get the resource faster and your server doesn't handle that traffic. You can control this behavior via the "Internal Data Download Type" option.
You can either keep your container/bucket private and rely on computation of a time-based signed URL (recommended) or you can make the whole bucket public and redirect to a public URL. The latter option is better for providers that also offer CDN, which is strictly path-based (such as DigitalOcean Spaces). Using private container/bucket and temporary signed URLs is preferred as it still keeps attachment permission checks relevant.
Non-attachments
Problem: internal_data/ is used not just for attachments but also for... other internal data. Specifically, there is code_cache/, file_check/, image_cache/, oembed_cache/, sitemaps/, temp/ and the beloved install-lock.php. My subjective opinion, personal preference and years of various experiments show that putting those other directories on a higher-latency is not a good idea (it will slow things down and occasionally throw errors because object storage is not your local filesystem).
Solution: This addon features an adapter-proxy that essentially handles different paths under the internal_data/ filesystem differently. It's enabled by default and you can control it via "Enable cloud filesystem for internal_data only for attachments" option. There is a use case where you might want to disable that, e.g. running XF on multiple containers/machines concurrently.
Attachment cache control
Problem: Images embedded as attachments are specifically never cached by the browser. This is just how XF works, but this addon provides a workaround.
Solution: This addon can pass-through (instead of redirecting to) images that are embedded as attachments while specifying Cache-Control: public and expiration according to "validity of temporary URL" (this option value is reused but in hindsight it should be a separate option).
The goal is two-fold:
Configuration options
License
This addon is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. You cannot use it for commercial purposes.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Yeah, seriously, you can pay as little as $1 a month to store 100 GB of attachments that your users will happily upload as soon as you let them.
Under the hood, this addon allows you to mount internal_data/ directory (or "filesystem", since it's powered by the flysystem) on cloud storage. There is an official guide that shows how to do it the hard way, but you might as well install this addon instead. Below you'll find a more detailed comparison between those two methods.
This addon does NOT change, in any way, the other "filesystem" called data/. It usually holds image attachment thumbnails, avatars and videos. There may or may not be a subsequent version that provides some capabilities around that space.
I am releasing this addon for free, in hopes that it will be useful to you, without warranty or support of any kind.
Features
- OpenStack Swift API support
- S3-compatible support
- Traffic offloading (redirect to attachment)
- Attached image passthrough to optimize caching
- Per-group attachment count and size limit (see permissions)
Comparison with mounting a Flysystem adapter manually
Excessive byte shoveling
Problem: Simply mounting a Flysystem adapter as internal_data/ causes every file operation to behave as if the cloud storage was a local filesystem. That means that every download is streamed from cloud storage, through your server to the end user. You would likely be paying TWICE for the storage and nullifying the effect of any CDN.
Solution: This addon attempts to redirect the user to the public or private (recommended) URL to fetch the attachment, so that they get the resource faster and your server doesn't handle that traffic. You can control this behavior via the "Internal Data Download Type" option.
You can either keep your container/bucket private and rely on computation of a time-based signed URL (recommended) or you can make the whole bucket public and redirect to a public URL. The latter option is better for providers that also offer CDN, which is strictly path-based (such as DigitalOcean Spaces). Using private container/bucket and temporary signed URLs is preferred as it still keeps attachment permission checks relevant.
Non-attachments
Problem: internal_data/ is used not just for attachments but also for... other internal data. Specifically, there is code_cache/, file_check/, image_cache/, oembed_cache/, sitemaps/, temp/ and the beloved install-lock.php. My subjective opinion, personal preference and years of various experiments show that putting those other directories on a higher-latency is not a good idea (it will slow things down and occasionally throw errors because object storage is not your local filesystem).
Solution: This addon features an adapter-proxy that essentially handles different paths under the internal_data/ filesystem differently. It's enabled by default and you can control it via "Enable cloud filesystem for internal_data only for attachments" option. There is a use case where you might want to disable that, e.g. running XF on multiple containers/machines concurrently.
Attachment cache control
Problem: Images embedded as attachments are specifically never cached by the browser. This is just how XF works, but this addon provides a workaround.
Solution: This addon can pass-through (instead of redirecting to) images that are embedded as attachments while specifying Cache-Control: public and expiration according to "validity of temporary URL" (this option value is reused but in hindsight it should be a separate option).
The goal is two-fold:
- it will allow the browser to cache it locally, so someone simply refreshing a page won't download them every single time;
- any edge cache in front of your forum (such as CloudFlare) will also cache that image and not redownload it from your server (this should offset the cost of proxying image through your server).
Configuration options
License
This addon is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. You cannot use it for commercial purposes.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
