Aug 11

Amazing Robocopy

There was a time when my home server was a large computer with RAID drives and loud fans that stayed on 24/7. Gladly that time is no longer, and I’ve moved to a more power-friendly netbook-based thing which generates hardly any heat and can therefore be left in a small cupboard without airflow concerns. The disk performance is, naturally, horrific, but it serves files up fast enough for me to stream music around the house.

I now do pretty much all my photo editing work on my laptop, which I take around with me.

As I still have the requirement for archiving large volumes of data to the desktop PC with hardware mirrored drives, unfortunately this machine has to stay. But I can keep it turned-off for 95% of the time, and only turn it on to copy / archive my data to it when needed. As well as saving energy, this should increase the lifespan of my disks by a long way too.

My netbook also acts as a web server from which I am hosting a simple ASP.net application to send WOL (wake on LAN) magic packets to my desktop PC. This is secured using Basic Authentication over SSL. It is therefore now possible for me to securely start my computer up when I’m away from home, then log on remotely using RDP (or establish a VPN connection). However – this is unrelated to my post.

I tend to do the large file transfers when at home, and given the regularity of these transfers, I’ve set up a directory on my main laptop into which I can dump everything ready for transferring, then run a robocopy batch file to move this data across.

Cue: “Amazing Robocopy“. (This is a super batch file which runs the robocopy routine and does a load more. I may have wasted an entire evening writing this batch file.) Hopefully the above gives you the context you need to fully appreciate why I need such a thing.

“Amazing Robocopy” does the following:

  • Checks if the remote machine is on. If not, send a WOL Magic Packet
  • Keeps pinging the remote machine until it responds, then attempts to check the fileshare (CIFS) visibility
  • Verifies some level of stability to the network connection before continuing
  • Run the robocopy and log to a file.
  • Provide the prompt at the very beginning as to whether user wants to shutdown machine when done
  • Provide feedback throughout the process
  • All timeouts and response limits set using variables
The screencap says it all:


Amazing Robocopy goes something like this.

It may be best to paste this into notepad before attempting to read.

Notes: download Depicus Wake On LAN for command line, and put it somewhere in your PATH. You’ll need to set the WOL arguments manually, as they can’t be configured with variables.

Also you’ll need to set the robocopy parameters to suit you.


: This script does a basic robocopy, but also it does the following:
: - test connectivity to the machine (ping). Send WOL Magic Packet if it doesn't respond.
: - after WOL, wait until the machine appears on the network
: - regardless of whether or not we got the machine up using WOL, we still verify a level of ping response consistency before continuing
: - verify the network path is visible before continuing
: - prompt at the start whether you want to shutdown the machine when finished
: - no further prompts during the process



: Set robocopy destination into two variables. They are used individually to test CIFS and PING connectivity then combined to insert into robocopy command
: We'll strip quotes from the outsides of these, so feel free to use quotes around each varilable - or not. 
Set remotemachine=mat-pc
Set copytoshare="f$\transfer\in"

: Time to wait after sending wol packet, before bothering to try to do anything else (approx startup time of remote machine)
Set timetowol=30

: If, after sending wol and waiting, there's still no response, we'll wait 1 second and try again.
: This is the total number of tries. TBH, may as well set this really high and Ctrl-C if you get bored.
Set pingfaillimit=25

: What do you consider is a good number of ping receipts to get back before deeming your connection to remote machine is stable? 1 = impatient. 10000 = paranoid. 10 = normal.
Set stabilitysatisfaction=10

: Once stability, by the definition of how many pings specified, is attained, we check the copy-to network path is available
: Frankly, if it isn't, it probably won't become available. And you'll have to figure out the problem separately.
: But this gives us the option to keep trying x number of times before continuing.
: Note: this number doesn't correspond to an amount of time. Windows is unpredictable when trying to check fileshares.
Set filesharefaillimit=15


CHOICE /M "Shutdown when done?
IF ERRORLEVEL 1 SET copyshutdown=1
IF ERRORLEVEL 2 SET copyshutdown=0

Set consecutivepingcheckcount=1
Set consecutivepingfailcount=1
Set filesharetestcount=1


ping /n 2 %remotemachine% | find "TTL=" >nul
if %errorlevel% == 0 goto reply

@echo No Reply on that IP! Tried %consecutivepingfailcount% of %pingfaillimit% times

IF %consecutivepingfailcount% == 1 (
@echo Let's try to WOL...

wolcmd.exe 001320164898 9
@echo OK ... WOL Magic Packet was sent. Let's wait for %timetowol% odd seconds then try to ping again...
ping -n %timetowol% >null
@echo Fine - let's try to connect now.
Set consecutivepingfailcount=1

Set consecutivepingcheckcount=1
Set /A consecutivepingfailcount+=1

IF %consecutivepingfailcount% == %pingfaillimit% (
@echo We didn't get very far did we?
@echo I sent a WOL, waited, but nothing!
@echo Increase the pingfaillimit variable?
GOTO fin
goto pingandcheck

@echo IP Replied! Checking connection stability... %consecutivepingcheckcount% of %stabilitysatisfaction%
Set /A consecutivepingcheckcount+=1
IF %consecutivepingcheckcount% == %stabilitysatisfaction% (
@echo Connection appears stable!
GOTO checkfileshare
GOTO pingandcheck

@echo Now checking fileshare
IF EXIST \\%remotemachine%\%copytoshare% (
@echo Fileshare is visible. Good to go. Starting copying.
GOTO docopy
@echo Couldn't find fileshare - tried %filesharetestcount% of %filesharefaillimit% times.
Set /A filesharetestcount+=1
ping -n 2 >null
IF %filesharetestcount% == %filesharefaillimit% (
@echo Failed to find the fileshare. Oh no!
@echo Maybe verify the fileshare is accessible yourself?
GOTO fin
goto checkfileshare

:first three lines strip quotes if found then combine the machine name and share to give path
for /f "useback tokens=*" %%a in ('%remotemachine%') do set remotemachine=%%~a
for /f "useback tokens=*" %%a in ('%copytoshare%') do set copytoshare=%%~a
Set destination=\\%remotemachine%\%copytoshare%
robocopy f:\transfer\out\ "%destination%" /E /R:20 /W:10 /MOVE /NP /LOG+:logfile.log /TEE /XF *.bat *.log

IF %copyshutdown%==1 (
ECHO off
shutdown /m \\mat-pc /s /f /t 0 


Aug 10

Roasting my own coffee beans

I used to be able to buy that Lavazza ground stuff (or Illy, or whatever), put it into a stovetop, and drink it with warm milk.

How things have changed. I now freshly grind my coffee (with a Baratza Vario coffee grinder) no more than 30 seconds in advance of the espresso extraction. It’s the only way to get that freshness and crispness I’m used to. I use a Rancilio Silvia to make my poison, and I can only buy my beans from Monmouth Coffee, where I know they have roasted them within at least the last few days.

The reason is that the beans deteriorate after about 10 days after roasting, in a way that is noticeable in the cup. I’m not a snob, I just have picky tastebuds. I can’t help that.

(It always amuses me, therefore, when people talk about the relative merit of one type of coffee over the other, in a supermarket. How any of it can be considered good is beyond me, when the stuff has been on a shelf for months. I guess it serves a purpose, but – yuk.)

The problem:

  • Freshly roasted beans – really fresh – are expensive in the first place
  • They are more expensive when you throw them away because they are no longer fresh and you didn’t get through them all quick enough
  • Getting freshly roasted beans (really fresh) is costly – postage from mail-order, a trip into central London
  • The beans are never there and fresh, when I want them.

A for-instance:

  • I buy 750g of beans because I made a special visit to Monmouth Coffee, use 125g, then because life is unpredictable I have to go away for 7 days then the remaining beans get thrown away (ack). Then I get back … to no beans (double ack). I have to wait at least a few days before I can get fresh stuff again (ack). During that time I have to visit cafés three times a day. Wow. A LOT of wasted beans, time, and money.

That’s 9 shades of annoying. And 4x ack.

I go away, I come home, I want fresh beans, right here, right now!

Cafés don’t have this problem. They get through so many beans each day they can afford a shipment every day. If they overorder / undersell one day, they can use rollover beans the next day, and reduce the order the following day. Beans always fresh. Not so chez Mat.

The solution:

Buy a home roaster. Store my own green beans for 6 months. Cheap! Fresh!

I have no desire to roast my own beans other than to get around the terrible feeling of coming home to no coffee.

Actually, that’s a lie. I love the thought of defining my own roasting profiles, choosing the type of roast according to my mood, experimenting.

So I am the proud new owner of a Behmor coffee roasting machine. None of this pokey air roasting rubbish. A proper drum roaster thank you. Complete with chaff and smoke management systems.

I’ll post back when I get some results from this thing!