Its_All.Lost

Because hope just breeds everlasting misery…

OS X and transferring files (with Transmit)

In my last post I mentioned how one of my server hard drives died and I ended up purchasing a replacement. I had done some thinking on what I wanted to do with regards on how to utilize the new drive, and that ClearOS 7 was now available. I decided to use the new 3TB drive as added storage, and migrate some files. I wanted to merge my Plex media to one source instead of being spread over three drives like it was. To do this was very easy, just time consuming due to the amount of movies I needed to move, somewhere around 450GB worth. I didn’t even think of using the terminal and issuing a “mv” command as I am sure it would have been faster and this issue wouldn’t have occurred. I fired up Transmit FTP and opened two windows. One window was the old drive and the other the new. I simply drag and dropped the files, copying them, and let it run while I went to sleep. I awoke the next day to an error of no free disk space on my Mac Pro. WTF? Upon investigation I found that Transmit was caching the file transfers locally on my Mac. Thats just fucking stupid. So I’m assuming that you cannot transfer more files than you have free disk space, because apparently it will run your drive down and crash your shit.

A reboot will clear the cache (for Transmit) or you can manually delete the cache. Its located at:
~/Library/Cache

There is a ton of other crap in there as well. Most of it should be safe to delete, it is just cache right?

Hard drive failure and adventures in ClearOS 7

I woke up Friday morning to the wife, “hey wake, the computer in the closet is making noises” (I have the server in the family room front entry closet with the door open). Oh great, fantastic way to wake up. Sure as shit it was making noises, zzzzzzzz-click zzzzzzz-click. Thing is toast. Good news or bad news first? Good? It was only a 160GB SATA 3GB/s drive nothing massive, and it was not the OS so the intarwebs and therefor our lives are all still functioning. The bad… it was a storage drive, a rather old storage drive – that was not backed up as it was rarely used. It contained all of our digital family pictures for the last 15 years. From the birth of our first daughter to the camping trip we had this last July. Gone. I never did like digital pictures.

In the wake of the disaster I ran out to Fry’s and purchased a replacement drive. Wow, drives are cheap now. I haven’t bought a hard drive in ages it seems. I was able to snag a 3TB 7200RPM SATA 6GB/s for $99 (without shopping around). 1TB drives as cheap as $40 bucks, damn! Even the 2TB hybrid-SSD drive was only $100 dollars.

Here is where the fun really begins. Since I had a shiny new, and massive, drive should I just do a fresh install and migrate to the new drive? Should I just toss it in as storage and rock on. Or, should I do a fresh install of ClearOS 7 that just recently dropped? I opted for a clean ClearOS 7 install. Sadly I feel it still hasn’t come very far from when I tried it back in Beta. It installed flawlessly this time and in a fairly decent amount of time. No issues with the quad-gigabit PCI-e network adapter which was nice. The thing that I noticed overall that worked out of the box with zero issues was the server sending email. My ISP blocks port 25 and I found ClearOS 6 to be a pain in the ass to get it to send mail, in fact I was unsuccessful in my attempts (I do not want to use my ISP as a relay). With 7 I was able to change the settings very easily to use GMail servers with no problems. However but it was not by any means a smooth configuration process. In fact I write this from my ClearOS 6.6 network, I booted back into the old drive (I didn’t wipe it!). I had nothing but problems trying to get Flexshares to work over the network, mainly because Samba was being a bitch. OpenLDAP and Samba were not playing nice. I am gathering from all the research and tweaking conf files that I installed some modules in an “incorrect” order. I found a post somewhere that mentioned the SMTP module was required to get Flexshares working 100%. WTF?

Anyways after some thinking I decided what I was going to do. I am currently transferring all my Plex media to the new 3TB drive where it will live. I am going to use one of the 80GBs as a torrent scratch disk (to prevent future catastrophes), and the other 80GB or 160GB as the new ClearOS 7 drive. Then I’ll migrate whats left to the 3TB for storage and wipe the other drives. Seven in all, about 5TB of space. I am thinking of playing with ClearOS 7 as a virtual box to get it right before getting dirty. I have one year before 6 goes EOL for support/updates. No problem.

Double your speed double your fun

I was at one of my in-laws neighbors homes the other day, he needed some help setting up Rabbit.TV and his Chromecast. I had never heard of Rabbit.TV, while it is cheap, $10 for the year, I found it very hokey and very difficult to navigate. Plus I believe it just links you to all the free content, but places it all in one convenient spot – I don’t know and not the point.

When I was all finished he asked if I wanted an extra cable modem he had. He had been mailed the modem in error or ended up not needing it after he bought it, I can’t remember which. I hesitated at first as I really need to stop collecting ewaste, but I took it and I am glad I did. I looked it up when I got home, nothing fancy just a Motorola SB6141. I was using a Ubee DDM3513 at the time. I originally has some Motorola Dinosaur but it had died, the Ubee is what was given to us in its stead. I haven’t had a lot of issues but I do feel it was sluggish at time. Not evidence to back it up thats just the overall feeling I was given from The Family. They two modems are both DOCSIS 3.0 the difference is (that would make a difference) that the SB6141 can handle “channel bonding of up to eight downstream channels and four upstream channels” (up to 300 Mbps downstream, 100 Mbps upstream). The Ubee is 4/4 (up to 171Mbps downstream, 122Mbps upstream). I Googled the hell out of the Motorola for a few hours and I came up with a lot of posts and forum comments on the performance. Most people advertised they were getting a higher Mbps with the SB6141 than other modems for their current speed tier (hitting 12Mbps on a 6Mbps account for example). So I decided to swap the modems out.

It actually went very smoothly, on Comcast’s side. I connected the new modem (SB6141) in place of the Ubee and plugged my netbook directly into it. It connected to the net within about 3 minutes or so. When I transferred the modem to the server/router is when I hit a snag. My ClearOS box wouldn’t grab a new IP address from the modem. If I watched it closely I could see it get the local 192.168.100.10 address, then drop it, then switch to the ISP public and then drop it 10 seconds later. Then I got to stare at the spinning loading logo forever. The netbook would grab it no problem every time. I power cycled the modem and the server/router many times, made no difference. I even tried cloning the MAC of the netbook on the server/router, it did take the clone but it made no difference either. What worked for me was to delete the ethernet card under “IP settings”. Mine is eth11 so I just deleted eth11 and then added it back in again. Voila it worked.

Speed tests ran from speedtest.net and speakeasy both report 12Mbps down, 1Mbps up. Prior to the modem switch I was only hitting 6.5Mbps, which is my speed tier cap. So 12Mbps on a 6Mbps speed tier – score, at least for now.

Update 10-22-2015: I have come to find an easier solution (other than not using Comcast and therefor not needing a cable modem). If you SSH into the box you can simply release and renew the IP of the ethernet port.

sudo dhclient -r your_nic followed by
sudo dhclient your_nic

For example my card is enp3s0 so I would type:

sudo dhclient -r enp3s0

That should release and renew the IP for you. I found this out after upgrading to ClearOS 7. The old trick of deleting and re-adding the NIC wasn’t working anymore. So it appears that it is not just Comcast and their cable modems, ClearOS has something to do with it.

ClearOS and FTP

Getting that same problem? Well apparently you are not alone, seems ProFTP and ClearOS don’t really work well out of the box. FileZilla would constantly timeout on listing the directories, so it would never actually connect. Transmit would timeout too, but it would spit out different errors, usually Error 162 Port Fail. After hours of digging I did find the problem, and the solution. It was actually pretty easy to fix (makes you wonder why the fix is not implemented?). Depending on your ClearOS setup YMMV and other steps may be required. I am running my ClearOS box in gateway mode, it is my server and router.

Start by forwarding all those ports required. Clear uses different ports for different things with FTP.
21 standard ftp port (internet say this is for home folders, but it connects me to flexshares). 60,000-61,000 are used for port 21 PASSIVE connections 2121 standard ftp port (internet says its for flexshares). 65,000-65,100 are used for port 21 PASSIVE connections. 989/990 SFTP ports.

To ditch the unroutable address warning toss this in your proftpd.conf file:
sudo nano /etc/proftpd.conf MasqueradeAddress ftp.mydomain.com  # using a DNS name

This did allow me to finally connect with Transmit with no issues. A friend of mine was able to connect with FileZilla after I made the changes. I had to change my FileZilla settings to use an Active connection versus a Passive or mine would refuse to connect. Odd.

I found these sites to be helpful, and thats where I got my information.
http://www.proftpd.org/docs/howto/NAT.html
https://www.clearos.com/clearfoundation/social/community/remote-ftp-access-through-nat-router-solved
https://www.clearos.com/clearfoundation/social/community/ftp-problem
http://forum.slicehost.com/index.php?p=/discussion/4491/ftp-error-162-port-failed/p1
http://www.theserverpages.com/articles/servers/cpanel/tweaks/Getting_passive_FTP_connections_to_work_through_a_firewall_properly.html
https://forum.filezilla-project.org/viewtopic.php?t=7315

ClearOS – broken system updates, thanks hal

So during my Plex not working NIC rearranging fiasco I was doing some digging to try and find the cause. It may have been related to some broken system update, not 100% sure. I did do a system update the other day, which had some broken dependencies,

This was the command that updated my broken hal dependency and got my system updated. Apparently the system was not configured to update with the clearos-core? Anywho, this did the job, and as soon as the update was finished Plex was working.

yum --enablerepo=clearos-core,clearos-contribs upgrade hal* libvirt* yum -y update

After doing a bit of searching I came up on this post that had the answer. I find the ClearOS forums search function to be horrible, and Google doesn’t seem to pick up on it much. Maybe this helps?