Torrent Invites! Buy, Trade, Sell Or Find Free Invites, For EVERY Private Tracker! HDBits.org, BTN, PTP, MTV, Empornium, Orpheus, Bibliotik, RED, IPT, TL, PHD etc!



Results 1 to 4 of 4
Like Tree5Likes
  • 3 Post By HGWLL
  • 1 Post By DGM
  • 1 Post By GRiP

Thread: Seedbox and Unlimited Amazon Cloud Drive Tutorial

  1. #1
    Senior Member
    HGWLL's Avatar
    Reputation Points
    2144
    Reputation Power
    54
    Join Date
    Feb 2016
    Posts
    330
    Time Online
    14 d 2 h 32 m
    Avg. Time Online
    6 m
    Mentioned
    75 Post(s)
    Quoted
    42 Post(s)
    Liked
    196 times
    Feedbacks
    15 (100%)

    Seedbox and Unlimited Amazon Cloud Drive Tutorial

    As promised here isa tutorial for a Seedbox with Amazon Cloud Drive as a backend.

    I have done it intwo parts, Part 1 is seedbox setup from scratch. Part 2 is Amazon Cloud Drivesetup.
    If you dont need tosetup a seedbox from scratch skip to Part 2.

    This tutorial willstart from a fresh install of Linux. This was tested on a dedicated serverrunning Ubuntu Linux 14.04.3 hosted by Online.net

    A note on Amazoncloud drive mounting, you need to make sure that if your using a VPS it's notOpenVZ or the kernel module for FUSE is enabled by your host.

    Be sure to replace<<USERNAME>> with your account username without << >>

    Part 1 -Seedbox Installation.

    SSH into yourserver.

    Update linuxlibraries:

    sudoapt-get update
    sudoapt-get upgrade

    Install somerequired files:

    sudoapt-get install subversion build-essential automake libtool libcppunit-devlibcurl4-openssl-dev libsigc++-2.0-dev unzip unrar curl libncurses5-dev
    sudoapt-get install apache2 php5 php5-cli php5-curl
    sudoapt-get install libapache2-mod-scgi
    sudoapt-get install screen apache2-utils

    We need to createsome directories in our user directory for rtorrent:

    sudomkdir /home/<<USERNAME>>/rtorrent
    sudomkdir /home/<<USERNAME>>/rtorrent/.session
    sudomkdir /home/<<USERNAME>>/rtorrent/downloads
    sudomkdir /home/<<USERNAME>>/rtorrent/watch
    sudochmod 777 /home/<<USERNAME>>/rtorrent/.session
    sudochmod 777 /home/<<USERNAME>>/rtorrent/downloads

    Create an installdirectory for downloaded installation files:

    sudomkdir /install

    Download and installxmlrpc

    cd/install
    sudo svn co http://xmlrpc-c.svn.sourceforge.net/svnroot/xmlrpc-c/advanced xmlrpc-c
    cdxmlrpc-c
    sudo./configure --disable-cplusplus
    sudomake
    sudomake install

    Download and installlibtorrent

    cd/install
    sudo wget http://rtorrent.net/downloads/libtorrent-0.13.6.tar.gz
    sudotar xvf libtorrent-0.13.6.tar.gz
    cdlibtorrent-0.13.6
    sudo./autogen.sh
    sudo./configure
    sudomake
    sudomake install

    Download and installrtorrent

    cd/install
    sudo wget http://rtorrent.net/downloads/rtorrent-0.9.6.tar.gz
    sudotar xvf rtorrent-0.9.6.tar.gz
    cdrtorrent-0.9.6
    sudo./autogen.sh
    sudo./configure --with-xmlrpc-c
    sudomake
    sudomake install
    sudoldconfig

    Download and installruTorrent

    sudo wget https://bintray.com/artifact/download/novik65/generic/ruTorrent-3.7.zip
    sudounzip ruTorrent-3.7.zip -d rutorrent
    sudomv ruTorrent-master rutorrent
    sudomv rutorrent /var/www/html
    sudochown -R www-data:www-data /var/www/html/rutorrent

    Setup rtorrentconfig files

    cd/home/<<USERNAME>>/
    sudonano .rtorrent.rc

    copyin the following:

    # Directories
    directory = /home/<<USERNAME>>/rtorrent/downloads
    session= /home/<<USERNAME>>/rtorrent/.session

    #rTorrent
    encryption= allow_incoming,try_outgoing,enable_retry
    schedule=watch_directory,5,5,"load_start=/home/<<USERNAME>>/rtorrent/watch/*.torrent"
    max_downloads_global= 0
    max_uploads_global= 0
    min_peers= 100
    max_peers= 300
    min_peers_seed= 100
    max_peers_seed= 300
    max_uploads= 350
    download_rate= 0
    upload_rate= 0
    check_hash= no

    #Network

    scgi_port= 127.0.0.1:5000
    encoding_list= UTF-8
    system.umask.set= 022
    port_range= <<YOURPORTRANGE>>
    port_random= no
    use_udp_trackers= yes
    dht =disable
    peer_exchange= no

    change the<<USERNAME>> parts to your username
    change port_range toa value of your choice.

    Create passwordprotection for ruTorrent

    sudohtpasswd -c /home/<USERNAME>/.htpasswd <USERNAME>

    You will be promptedfor a password remember what you choose as it will be used to login to thewebserver.

    Create SSLcertificate to protect webserver connections

    sudoa2enmod ssl
    sudoservice apache2 restart
    sudomkdir /etc/apache2/ssl
    sudoopenssl req -x509 -nodes -days 9999 -newkey rsa:2048 -keyout/etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt

    You will be promptedfor some info, you can just press enter for some but the following areimportant.
    If you are usingonline.net like me the settings are:

    Country:FR
    Province:Īle-de-France
    City: Paris
    CommonName: <SERVER IP ADDRESS HERE!!>

    Configure webserver(Apache) to force SSL and use our generated SSL Certificate.

    sudonano /etc/apache2/sites-available/000-default.conf

    Change so itcontains only:

    <VirtualHost*:80>
    ServerName<<SERVER IP HERE>>
    Redirect permanent / https://<<SERVER IP HERE>>/
    </VirtualHost>

    sudonano /etc/apache2/sites-available/default-ssl.conf

    Find the followingand change to match:

    <VirtualHost<<SERVER IP>>:443>
    SSLEngineon
    SSLCertificateFile/etc/apache2/ssl/apache.crt
    SSLCertificateKeyFile/etc/apache2/ssl/apache.key

    And add thefollowing just before the </VirtualHost> tag at the end of the file.

    <Directory/var/www/html/>
    Options-Indexes
    AllowOverrideALL
    </Directory>

    <Location/>
    AuthName"Private"
    AuthTypeBasic
    AuthBasicProviderfile
    AuthUserFile/home/<<USERNAME>>/.htpasswd
    Requireuser <<USERNAME>>
    </Location>

    <Location/rutorrent>
    AuthName"Private"
    AuthTypeBasic
    AuthBasicProviderfile
    AuthUserFile/home/<<USERNAME>>/.htpasswd
    Requireuser <<USERNAME>>
    </Location>

    SCGIMount/RPC2 127.0.0.1:5000
    <Location/RPC2>
    AuthName"rTorrent secure access"
    AuthTypeBasic
    AuthBasicProviderfile
    AuthUserFile/home/<<USERNAME>>/.htpasswd
    Requireuser <<USERNAME>>
    </Location>

    sudoservice apache2 restart
    sudoa2ensite default-ssl
    sudoservice apache2 reload
    sudoservice apache2 restart

    Make sure all filesand folders in our home directory belong to our user:

    sudochown -R <<USERNAME>>:<<USERNAME>>/home/<<USERNAME>>

    Start rtorrent usingscreen:

    screen-fa -d -m rtorrent

    You should now beable to open a web browser and navigate to:

    https://<SERVERIP>/rutorrent

    and download/uploadtorrents.
    When prompted enteryour username and the password you chose.


    We need to configureSSH:

    sudonano /etc/ssh/sshd_config

    Change

    Port to something different
    PermitRootLogin to no

    Add

    AllowUsers<<USERNAME>>
    AllowTcpForwardingyes

    Restart SSH

    sudoservices ssh restart

    Now lets setup anFTP Server so we can download/upload files, I use ProFTP but you can substitutefor another if you like.

    sudoapt-get install proftpd

    Choose Standalonewhen given the option.

    We need to configuresome options:

    sudonano /etc/proftpd/proftpd.conf

    Change

    ServerName to name of your choice.
    Port to something else to minimize port scanning.
    DefaultRoot~/rtorrent/

    Add the following toend of file:

    <LimitLOGIN>
    AllowUser<USERNAME>
    DenyAll
    </Limit>

    Next edit/createthis file to enable SFTP capabilities:

    sudonano /etc/proftpd/conf.d/sftp.conf

    Add the following:

    <IfModulemod_sftp.c>
    SFTPEngineon
    Port<PORT OF CHOICE HERE>
    SFTPLog/var/log/proftpd/sftp.log

    #Configure both the RSA and DSA host keys, using the same host key
    #files that OpenSSH uses.
    SFTPHostKey/etc/ssh/ssh_host_rsa_key
    SFTPHostKey/etc/ssh/ssh_host_dsa_key

    SFTPAuthMethodspassword
    #SFTPAuthMethodspublickey

    #SFTPAuthorizedUserKeysfile:/etc/proftpd/authorized_keys/%u

    #Enable compression
    SFTPCompressiondelayed
    </IfModule>

    Change PORT OFCHOICE, this is the port you will SFTP into.
    Restart ProFTP withnew configuration.

    sudoservice proftpd restart


    Installing PlexMedia Center (OPTIONAL):

    cd/install
    wget https://downloads.plex.tv/plex-medi...xmediaserver_0.9.12.13.1464-4ccd2ca_amd64.deb
    sudo dpkg –iplexmediaserver_0.9.12.13.1464-4ccd2ca_amd64.deb

    To configure PLEXyou will need to connect to your server through SSH as a proxy.

    If you are usingwindows configure putty as follows:
    Host Name is yourservers IP
    Port is the Port youchose for SSH connections
    Connection Typeshould be SSH

    In the Side menu youselect Connection -> SSH -> Tunnels

    Make source port6666
    Select Dynamic andAuto

    Click Open and loginto your sever with your username and password.

    Now download/openFirefox

    Press Alt on yourkeyboard, select Tools -> Options.
    Select Advanced-> Network
    Under Connectonsselect Settings
    Select Manual proxyconfiguration

    SOCKS Host =localhost
    Port = 6666

    And deletelocalhost, 127.0.0.1 from “No Proxy for:”

    Close settings andnavigate to:

    localhost:32400/web

    and configure PLEX.


    PART 2 -Amazon cloud drive mounting.

    Amazon Cloud Drive.

    We need to installsome dependancies

    sudoapt-get update
    sudoapt-get install python3-setuptools python3-appdirs python3-dateutilpython3-requests python3-sqlalchemy
    sudoeasy_install3 -U pip
    sudoapt-get install git

    Install acd_cli:

    sudo pip3 install --upgrade git+https://github.com/yadayada/acd_cli.git

    We now run

    acd_clisync

    This will pop upsome info don't press anything for now and goto the following link:

    https://tensile-runway-92512.appspot.com/

    You log in with youramazon cloud details and it will generate a file called oauth_data which needsto be placed in the directory

    /home/<<USERNAME>>/.cache/acd_cli

    I copied it to thisdirectory by logging in via SFTP and uploading the file and then copying to thedirectory above.

    if you exitedacd_cli to copy the oauth_data file run

    acd_clisync

    again and if itend's with ...done your good to go.

    Now we need tocreate five directories in our home directory, Ill explain each one as wecreate them

    cd/home/<<USERNAME>>

    mkdiramazon
    mkdiramazon/.local-enc
    mkdiramazon/local-enc
    mkdiramazon/.cloud-enc
    mkdiramazon/cloud-enc
    mkdiramazon/sorted

    .local-enc <-will hold encrypted files ready to be uploaded to the cloud
    local-enc <-files copied here will be encrypted and end up in .local-enc

    .cloud-enc <-will be where amazon cloud encrypted files are mounted
    cloud-enc <- willbe where amazon cloud drive encrypted files are decrypted and readable

    sorted <- will bewhere cloud-enc and local-enc will be overlayed together so we can maintain thesame directory structure between pending uploads and existing cloud contents

    Lets install encfsand unionfs-fuse

    sudoapt-get install unionfs-fuse encfs

    We now need tocreate encyption keys for encfs:

    encfs/home/<<USERNAME>>/amazon/.local-enc/home/<<USERNAME>>/amazon/local-enc

    Use the defaultsettings and enter a strong password, don't forget the password, without it youwill lose all your data.

    Unmount the drive wejust created:

    fusermount-u /home/<<USERNAME>>/amazon/local-enc

    We need to copy thefile containing all the encryption details out of the upload directory, MAKESURE YOU ALSO BACK THIS FILE UP OFF THE SERVER TOO.

    mv/home/<<USERNAME>>/amazon/.local-enc/.encfs6.xml/home/<<USERNAME>>/amazon/encfs.xml

    We need to remountand reference where we moved the encryption details too:

    ENCFS6_CONFIG='/home/<<USERNAME>>/amazon/encfs.xml'encfs /home/<<USERNAME>>/amazon/.local-enc/home/<<USERNAME>>/amazon/local-enc -o allow_other

    We now need to mountthe amazon cloud drive using acd_cli:

    acd_cli mount -ao/home/<<USERNAME>>/amazon/.cloud-enc/

    acd_cli sync

    No we can mountencfs over acd_cli to decrypt the files on amazon cloud drive:

    ENCFS6_CONFIG='/home/<<USERNAME>>/amazon/encfs.xml'encfs /home/<<USERNAME>>/amazon/.cloud-enc/home/<<USERNAME>>/amazon/cloud-enc -o allow_other


    Everything shouldnow be ready
    http://torrentinvites.org/0677CD33-D...F534BAEF68.png

    acd_cli now hassupport to write directly to amazon cloud drive (you can copy directly tocloud-enc) however I dont recomend that just yet, Im waiting till acd_cli isout of beta.

    Instead to upload tothe cloud drive copy the files/folders you want to local-enc or sorted and theywill be encypted to .local-enc then run:

    acd_cliupload -x 2 --overwrite /home/<<USERNAME>>/amazon/.local-enc/* /

    -x 2 tells acd_clito use 2 connections
    --overwrite tellsacd_cli to overwrite files that have a different file size or timestamp
    the / at the endtells acd_cli to upload to the root directory of the cloud drive if you wish tochange this then append to the end.

    If you get anyerrors run the same command again to correct the errors.

    To give a consistentfile structure between local-enc and cloud-enc:

    unionfs-fuse-o cow/home/<<USERNAME>>/amazon/local-enc=RW:/home/<<USERNAME>>/amazon/cloud-enc=RO/home/<<USERNAME>>/amazon/sorted/ -o allow_other

    Now the sorteddirectory will display both the local-enc and cloud-enc together. This is thedirectory that I give to Plex to catalogue.
    If you write to thesorted folder it will write to local-enc for upload. If you read from thesorted directory, it will check local-enc first and if the file doesnt existlocally it will read from cloud-enc.

    I have filebotcopy/rename files to /home/<<USERNAME>>/amazon/local-enc and thenupload from there.

    To delete files fromlocal disk once uploaded run:

    rm -rf/home/<<USERNAME>>/amazon/.local-enc/*

    You can createscripts to create automated processing, you can do alot with this setup, playaround with it.

    Have Fun!

    Just as a side note,if you use the cloud drive for a media server limit the amount of people youshare it with, you will be ok with a few friends and family, but huge amountsof traffic and amazon may notice and may disable your account.

    Credit to sources Ihave used:

    https://amc.ovh/2015/08/14/mounting-uploading-amazon-cloud-drive-encrypted.html
    https://amc.ovh/2015/08/15/uniting-encrypted-encfs-filesystems.html
    http://makeshift.ninja/index.php/20...media-server-w-amazon-unlimited-backend-pt-1/

    Check them out theygo into more detail than I have, I've added some things, and merged thingstogether.
    These sources havebeen majorly helpful to me as Im sure they will be to you.

    If I've missedanything or you spot something that needs correcting let me know
    Laxus, DGM and bhazeltn like this.

  2. #2
    DGMDonor Icon
    DGM is offline
    iLLuSioNist
    DGM's Avatar
    Reputation Points
    77147
    Reputation Power
    100
    Join Date
    Aug 2015
    Posts
    4,744
    Time Online
    204 d 20 h 52 m
    Avg. Time Online
    1 h 33 m
    Mentioned
    969 Post(s)
    Quoted
    453 Post(s)
    Liked
    4014 times
    Feedbacks
    170 (100%)
    @HGWLL nice tutorial & share. It is worth mentioning that Amazon suspend account with unusual/doubtful usages of their services.

    If anyone going use it then it is recommended that transfer torrent only which are not risky (if gone) n HnR already cleared.

    Like & Rep added. Keep it Up Mate. Looking forward for your next share.
    Laxus likes this.
    DGM Says ! Be Busy Be Happy TI'ian. !

  3. #3
    New user GRiP's Avatar
    Reputation Points
    10
    Reputation Power
    31
    Join Date
    Jan 2016
    Posts
    5
    Time Online
    4 h 45 m
    Avg. Time Online
    N/A
    Mentioned
    0 Post(s)
    Quoted
    1 Post(s)
    Liked
    2 times
    Feedbacks
    0
    thanks bro, will try it this weekend.
    HGWLL likes this.

  4. #4
    Donor
    bhazeltn's Avatar
    Reputation Points
    1010
    Reputation Power
    55
    Join Date
    Jan 2012
    Posts
    5
    Time Online
    6 h 36 m
    Avg. Time Online
    N/A
    Mentioned
    3 Post(s)
    Quoted
    2 Post(s)
    Feedbacks
    0
    @HGWLL Thanks for the guide!

    Following yours as well as the source guides (which I had found first) I have my setup working - mostly. I've had some issues with automation, but I think they stem from other issues.

    I have my seedbox set up how I need. Not exactly a typical setup, but it is working. Because I'm using Virtualmin for my personal hosting I had to do some work arounds. I used https://github.com/xombiemp/ultimate-torrent-setup/wiki as a template because I liked how it worked when I did everything locally.

    I'm using Emby on my local media server (still have 20TB of media on it that I will replace in the cloud) because I prefer how it integrates into Kodi.

    My problems are in this area:
    Sorting media and uploading to ACD

    Here is my folder set up for reference:
    /seedbox/.acd (acd encrypted)
    /seedbox/acd (acd plain view)
    /seedbox/.local/ (encrypted local)
    /seedbox/Media (unionfs-fuse mount of /seedbox/acd/Media and /seedbox/local/Media)
    /seedbox/local (local plain view)
    /seedbox/torrents/* (unencrypted folders for rtorrent to download to)


    I've tried this using 3 methods - both manually with some automation attempts.
    First was having Sonarr and Couchpotato rename and sort in an encryped folder on the seedbox. The sorting and renaming worked well. Uploading it to ACD manually worked ok as I would make sure there was nothing moving around before I started the process.

    2nd method was almost the same as above but instead of using /seedbox/local to do the work I was using /seedbox/Media so Sonarr could see the library after files were uploaded and deleted.

    Both these methods were problematic on automation. Partial files were being uploaded. acdcli was uploading everything instead of the specific file (which I will look at refining in the inotify-tools scripting). Trying to use the script on makeshift.ninja as a guide was problematic in uploading a single file with encryption.

    3rd was a little more complex.
    Torrents would download then copy to a folder on the encryped local - /seedbox/local/tv
    Would upload to /seedbox/acd/tv (acdcli and the encrypted mounts, of course)
    Sonarr is watching /seedbox/acd/tv and sorting to /seedbox/acd/Media/TV Shows/ which it is also looking at for the library.
    Sonarr seems to not like this setup either. It starts trying to rename and move files before they are uploaded (somewhat expected!) which ends up with corrupted data.

    Is there a 4th method I am missing or would fine tuning one of these methods work best? From what I can understand from what is happening I should be using some version of the first 2 methods with some fine tuning on the scripts.

    Any suggestions?



Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •