06 Dec 2013
Intro
I heard that the 37Signals guys wrote another book, this time about working remotely. While I didn’t read the book (nor do I intend to) I am definitely glad it’s out there, and if more people would work remotely following this book I’d be very happy.
You can say that this book was the trigger to this post :)
My reasons
1. Providing more value to my employer
When I am working remotely, I am more valuable. PERIOD. That is a fact b.t.w, and I will explain it as we go further down this post.
Since I am a consultant, I insist of providing the highest value possible for my clients, working out of their office during their working hours means I am less effective, so I just don’t do it.
I work to provide value, not working hours
2. Setting my own hours
Working remotely gives me the ability to work my own hours, and I only work when I am the most effective and the most productive. Now I realize that most people say that about remote work, but I said I will back it up with facts, so here are a few working days from the last week or so.
As you can see from this day, I worked after midnight the day before, then I did not even touch my computer until around 11am the following day.
I worked really productive until around 8pm where I just stopped working and started using my computer for personal things like Facebook etc…
Not your typical day working out of an office right. Well you could say that you have the possibility to arrive late and leave late. I would agree so lets look at a few more days.
Again, finishing the day before after midnight, worked for 25 minutes around 10am and then pretty much worked for 6 hours straight.
Stopped touching my computer around 3pm and then went back to work around 9pm working like crazy for 3 hours, being productive for 59 minutes each hour. That is GOLD.
This is a pretty typical day for me, I zone out around the early afternoon and then get back to work after putting the kids to sleep and though their night time routine.
One more?
This is quite a long day for me, usually I do this after the day before I didn’t work for long (because I got stuck with a problem or something).
As you can see I worked from 9am to 9pm pretty straight, fairly productive for such a long day.
The other side of the story
I would be biased if I didn’t show you a typical day working out of an office for me.
As you can see, the day is significantly shorted, even though theoretically you work for 9 hours, you have lunch in the middle, then you have a couple of meetings, then around 4-5pm you just crash from fatigue struggling you keep your focus.
This is pretty consistent for me working out of an office, the day starts too early and you don’t have time to recover so you work when you are fried most of the time.
Getting to the office at 9am leaves me with no time for myself or my family as well.
3. Only working when I am productive.
Taking out bugs or crashes, I don’t work when I am not productive, if I feel that I am zoning out too much, or that I write code that is not 100% up to my standards, I just don’t work.
I go out, I get some fresh air, I basically do anything that is not work, and usually completely off the computer, I try not to sink into Facebook or other sites, because then my brain doesn’t refresh as it should.
4. Peace and quiet
Office environment is a very loud very disruptive environment, you don’t really get to spends 3-4 hours of quiet time, alone with yourself, just coding, without anyone hovering over you with questions or problems.
Now, questions and problems are part of my day, and I really love them, but those should be async notifications, you should not be zoned out from your code or the problem you are working on.
Your focus is fragile, if you lose it, you might not get it back for another 20-30 minutes (just enough time for someone to ask another question).
That’s why, usually in offices, this is a common view:
Engineers are putting on HUGE headphones, just to try and zone out for a few minutes.
5. Your own daily routine
Since I can set my own hours here are the things I do that most engineers don’t
- I exercise
- I watch my diet very carefully
- I pick up my kids from kindergarten (1pm and 2pm) every day
- I eat lunch with my kids and my wife
- I spend time with my kids during the week
The base idea is that my life are not set by my work, it’s the other way around, I don’t do the family’s grocery shopping after work, when I am tired from a day’s work, it’s a clear recipe to buying crap you don’t need.
That’s only one example, but you can really imagine what this sort of flexibility gives you.
6. Exercise
I want to focus a bit more about exercise, as you can imagine I have this data as well :)
As you can see from my training schedule, I have been pretty consistent with my training, doing around ~24 hours per month until Sep where I peaked around 36.
Then, November through me off, mainly because I worked 2 weeks out of an office.
The hours killed me, since you can’t go out cycling when it’s dark you are limited on your training time, so it was really hard to find the time to train before work, and after work was already dark.
The importence of cycling (exercising) to me is huge, exercising puts my day in focus, those 2 hours I spend a day outside in the fresh air (even though I am suffering) really help me through the day.
I am more focused, less tired, less edgy, everything is better when you exercise.
Most people I know around me don’t do it, and I feel sorry about that, it’s such a great thing to do for people that sit around most days. (switch to a standing desk if you do)
Go outside, exercise, thank me later :)
What does it require
Self discipline
The one quality which sets one man apart from another- the key which lifts one to every aspiration while others are caught up in the mire of mediocrity- is not talent, formal education, nor intellectual brightness - it is self-discipline.
With self-discipline all things are possible. Without it, even the simplest goal can seem like the impossible dream. - Theodore Roosevelt
Self discipline is crucial to working remotely.
As you can see, I don’t work 3 hours a day, I don’t work in my PJ’s and I don’t slack off.
I get up, I have a routine and I focus on providing value.
I absolutely love what I do every day, this keeps me motivated to keep providing value.
Communication
Communicating clearly when you work remote can be challenging, it requires you to be descriptive, take the time to explain yourself and be clear about your intentions.
I communicate through chat and email most of the day, but when this isn’t sufficient, I am always a Skype or hangout away.
Summing up
I tried not to get into what it takes to make it work too much, I am going to write a post about it in the near future so I though i’d leave this open, at least partially for now.
You can see that I’ve been doing it for a long time, not once did I have a problem BECAUSE I was working remotely, with any company I worked for, not even once.
I believe that working remotely for me is the most effective way, if that wasn’t true, I wouldn’t be able to do it for so long.
Feel free to speak up in comments, I would love to know what you think.
03 Dec 2013
Lately, I needed to upload a folder to S3 with all of it’s files.
The use-case is compiling assets on the CI and then uploading it to S3 for the CDN to consume.
While searching for a gem that does it I encountered s3_uploader, but I really didn’t like it because it’s using Fog.
Generally, I don’t like gems that use other gems for no apparent reason, there’s absolutely no reason to include fog in my project just to upload files recursively.
I did however, like that it’s using multi threads in order to do the upload so I am doing the same in my solution.
I wrote a solution that uses the aws-s3 Ruby SDK, which was already included in my project anyway.
Here’s the code:
class S3FolderUpload
attr_reader :folder_path, :total_files, :s3_bucket
attr_accessor :files
# Initialize the upload class
#
# folder_path - path to the folder that you want to upload
# bucket - The bucket you want to upload to
# aws_key - Your key generated by AWS defaults to the environemt setting AWS_KEY_ID
# aws_secret - The secret generated by AWS
#
# Examples
# => uploader = S3FolderUpload.new("some_route/test_folder", 'your_bucket_name')
#
def initialize(folder_path, bucket, aws_key = ENV['AWS_KEY_ID'], aws_secret = ENV['AWS_SECRET'])
@folder_path = folder_path
@files = Dir.glob("#{folder_path}/**/*")
@total_files = files.length
@connection = AWS::S3.new(access_key_id: aws_key, secret_access_key: aws_secret)
@s3_bucket = @connection.buckets[bucket]
end
# public: Upload files from the folder to S3
#
# thread_count - How many threads you want to use (defaults to 5)
#
# Examples
# => uploader.upload!(20)
# true
# => uploader.upload!
# true
#
# Returns true when finished the process
def upload!(thread_count = 5)
file_number = 0
mutex = Mutex.new
threads = []
thread_count.times do |i|
threads[i] = Thread.new {
until files.empty?
mutex.synchronize do
file_number += 1
Thread.current["file_number"] = file_number
end
file = files.pop rescue nil
next unless file
# I had some more manipulation here figuring out the git sha
# For the sake of the example, we'll leave it simple
#
path = file
puts "[#{Thread.current["file_number"]}/#{total_files}] uploading..."
data = File.open(file)
next if File.directory?(data)
obj = s3_bucket.objects[path]
obj.write(data, { acl: :public_read })
end
}
end
threads.each { |t| t.join }
end
end
The usage is really simple
uploader = S3FolderUpload.new('folder_name', 'your_bucket', aws_key, aws_secret)
uploader.upload!
Since it’s using Threads, the upload is really fast, both from local machines and from servers.
Have fun coding!
22 Nov 2013
I work a lot with MongoDB, and this is not the “Why not MongoDB” or “Why MongoDB” type of post, I know it’s popular to trash Mongo lately.
Anyway, while working with Mongo, from time to time I need to export a collection, or a subset of the collection based on a query.
I love using mongoexport because you basically get JSON file out of it (or CSV) and from there on you can pretty much do anything you want with it.
You can use Amazon’s MapReduce or any other solution you may want.
For example, when I do usually is export a list I need, load it into Rails console and work with the output, queue it up to the worker list etc...
Let’s cover some scenarios I use the most
Export Collection To JSON
Actually, I never use it, since the collections are too big for the disk to handle at once, we have a sharded collection, so no one-disk solution can hold the data.
That been said, I think for most people this can be very useful.
mongoexport --host HOST --port PORT --db DB_NAME -u USERNAME -p PASSWORD --collection COLLECTION_NANE -q '{}' --out YOUR_FILENAME.json
Export part of the collection to JSON (using a query)
mongoexport --host HOST --port PORT --db DB_NAME -u USERNAME -p PASSWORD --collection COLLECTION_NANE -q '{ "some_numeric_field": { "$gte": 100 } }' --out YOUR_FILENAME.json
The most important part here is that the -q options needs to be a valid JSON format query that Mongo knows how to handle { "some_numeric_field": { "$gte": 100 } }. You can of course use far more complicated queries, but for most cases I don’t need to.
Bonus
I use Dash every day, multiple times a day, so it was only natural to have a dash snippet that I can use
Dash snippets are basically a way to paste some code using a shortcode, so typing mongoexport in the console pops up a window where I can complete the rest of the command easily without remembering the options.
Here’s the Dash snippets
mongoexport --host __host__ --port __port__ --db __db-name__ -u __username__ -p __pass__ --collection __collection__ -q '__query__' --out __filename__.json
And this is what the window looks like when I type the shortcode, I tab through the place holders and there’s a no-brainer way to remember the command and the options.
06 Oct 2013
We recently opened a couple of API endpoints at Gogobot to the public.
I was looking for a way to generate API documentation in a good looking way, one that will appeal to my fellow developers.
After a little while search with Google, I thought I had it nailed with a Twitter Bootstrap theme, but the way I had to generate the documentation was too manual, I had to edit lots of files and I really didn’t find it “easy”.
So, I was looking around some more, and I finally found a really easy way to generate API documentation here: apidocjs.com
It really nailed everything I was looking for:
- Super easy customizable template.
- Intuitive documentation
- Document versioning
- Version compare
Everything is super easy and at your fingertips.
One thing that I have to say badly about it, is that it’s using an old deprecated Markdown parser, but I am already working to fix it and post a Pull Request for it (Gotta love open source)
14 Aug 2013
I have been working with Sidekiq for quite a while now, having many jobs per day working (multiple millions of jobs.)
Sometimes, I queue up tasks to a queue called #{queue_name}_pending, I do this so I can manage the load on the servers. (For example: Stop writing to Mongo, Stop importing contact etc…)
This way, I can queue up many jobs, and I can move it to the real queue whenever I feel like it or whenever the problem is solved.
I was looking for a way to move tasks from one queue to another.
There’s nothing built into Sidekiq for this, but obviously, you can just use redis built in commands to do it.
Here’s the code to do it
count_block = proc{ Sidekiq.redis do |conn|
conn.llen("queue:#{queue_name}")
end }
while count_block.call > 0
Sidekiq.redis do |conn|
conn.rpoplpush "queue:#{queue_name}_pending", "queue:#{queue_name}"
end
end
This will move all the items from one queue to another until there are no more jobs.
b.t.w
Obviously, the _pending queues don’t have any workers assigned to them, the purpose of it is a place holder so the jobs won’t go to waste and we can resume work when we can.
26 Jul 2013
Usually, when I have to do something in my code, I add a TODO comment (and forget about it).
For example
class UserThank
module Scores
def _score
return Scoring::UserLeaderboard::WEIGHTS[:thanks]
end
# TODO: Should this be the user attached to the activity or should this be the thanked user?
# Ask product and change the code
def _receiver
[self.user]
end
end
end
I have the shortcuts to show all TODO in vim, but it’s clearly not in my muscle memory and not in any kind of memory, natural or computerized, I just don’t do it enough, so it adds up and I feel bad about it.
Today, while coding I encountered this case, where clearly it’s not just a simple thing, it’s not just something I as an engineer care about, it’s a product related thing and affects core user experience.
So, I think I found a better way to actually make sure I DO my TODO.
I thought to myself, what do I notice every single time, I never overlook…
The answer was clear, tests, if a test fails I will never overlook it, will never deploy if a single test fails, same goes for the rest of the team.
so here’s the solution
context "User Thank" do
it "should score the user" do
pending "This is pending some investigation on the product side, right now I will fail it" do
true.should be_true
end
end
end
I make use of a super useful Rspec feature, where you pass false or an exception into a pending block, it fails the spec.
I just describe the problem in detail, and that’s it, this is a piece of code that will be dealt with, before deploy.
19 Feb 2013
I started using MongoDB at Gogobot a little while ago.
While using it, I encountered some problems, but for the most part, things went pretty smooth.
Today, I encountered a bug that surprised me.
While it certainly should not have, I think it can surprise you as well, so I am writing it up here as a fair warning.
For Gogobot, the entire graph is built on top of MongoDB, all the things social are driven by it and for the most parts like I mentioned, we are pretty happy with it.
SO, What was the problem?
The entire graph is a mountable engine, we can decide to turn it on or to turn it off at will.
It acts as a data warehouse and the workflows are being managed by the app.
For example:
When model X is created, app is notified and decides what to do with this notification, and so on and so forth.
Everything peachy so far, nothing we haven’t used hundreds of times in the past.
Here’s how it works.
We have a model called VisitedPlace, it’s a representation of a user that visited a certain place
Here’s the code
module GraphEngine
class FbPlace
include Mongoid::Document
include Mongoid::Timestamps
include GraphEngine::Notifications::NotifiableModel
#… rest of code here
end
end
As you can see, this model includes a module called NotifiableModel, here’s the important part from it:
module GraphEngine
module Notifications
module NotifiableModel
extend ActiveSupport::Concern
included do
after_create do
send_notification("created")
end
end
def send_notification(verb)
# Notify the app here...
end
end
end
end
Like I said, pretty standard stuff, nothing too fancy, but here’s where it’s getting tricky.
This model has a unique index on user_id and place_id. It’s a unique index and no two documents can exist in the same collection.
BUT… check this out:
GraphEngine::VisitedPlace.create!(user_id: 1, place_id: 1) => true
GraphEngine::VisitedPlace.create!(user_id: 1, place_id: 1) => true
The second query actually failed in the DB level, but the application still returned true.
Meaning, that after_create is actually being called even if the record is not really persisted.
How you can fix? / should you fix?
For Gogobot, I fixed it using safe mode on those models, I don’t mind the performance penalty, since I don’t want to trigger Sidekiq workers that will do all sorts of things twice or three times.
Should you do the same? I am not sure, you need to benchmark your app and see if you can fix it in another way.
Would love to hear back from you in comments/discussion
04 Feb 2013
For the past few months I have been super busy building the social graph behind Gogobot.
In this presentation I gave in Israel I go through a fraction of the Gogobot platform and how I implemented a social feature
Feel free to comment & discuss.
Enjoy!
30 Jan 2013
A few weeks back, we started slowly upgrading all of our queues at Gogobot to work with Sidekiq.
Posts on how awesome the experience was and how much better Sidekiq is from Resque coming soon, though with all the good came some bad.
Summary of the solution
With Sidekiq, we are processing around 25X more jobs than what we were doing with Resque, processing around 15,000,000 jobs per day, at paces of over 1K per second at times (at peak we go up well past that)
This is how many jobs we processed today…
And this is a snapshot of our realtime graph
On the MongoDB side we are working with Mongoid and we have a shared environment, 9 shards with 3 replicas in each shard, all running through 2 routers.
Our production mongoid config looks like this
production:
op_timeout: 3
connection_timeout: 3
sessions:
default:
hosts:
- HOST_NAME:27017 #Single router R0
username: USER_NAME
password: PASSWORD
database: DATABASE_NAME
options:
consistency: :eventual
We are using latest versions of all relevant gems (Sidekiq, Mongoid, Moped, Redis)
All seems fine right? What’s the problem?
The problem is that we have too many connections opening and closing to our mongo instances. (~25-40 new connections per second).
Each time a job is picked up, a connection to Mongo is opened and when the job is done, this connection is closed (using Kiqstand middleware).
This is causing huge loads on our router server, and causing mongo to run out of file descriptors at times.
SO?
More then anything, this post is a callout for discussion with anyone using similar solution with similar scale and can assist, I know I would love to brainstorm on how to solve this problem.
13 Nov 2012
My slides from the talk I gave at RailsConf Israel 2012.
Enjoy!