Day 33: Choose your languages wisely.

STATUS: Welp, I wasted an entire day trying to write multi-threaded PowerShell code. In other words, I moved two steps forward and two steps back.
MOOD: I actually feel surprisingly alright with this.

——————

Let’s get technical for a second.

I am using PowerShell to create the MVP for thinglistr’s backend because it’s the language that I know most extensively right now. I’m also a C# and F# developer (I miss writing in F# A LOT), but I needed to get this thing in the air as quickly as possible, and PowerShell is much easier to write and test in a SSH session than F# (despite it having a console, `fsi`) or C# are, so I chose that. (Before you say “Y U NO PYTHON,” my Python knowledge pales hard in comparison to my PowerShell knowledge and I wanted to spend more time building than learning the semantics of a lanuage. I’m also pretty set in using C# or F# for production (F# is great for transactions and multi-threaded code; immutability by default helps a lot).

This setup has worked pretty well so far. Working with RESTful queries and caching things (primitively) is pretty straightforward, and it does the backend stuff well enough to satisfy my needs right now. That didn’t stop me from wondering whether I could make things a little faster, though. It would be much more efficient for the business polling and review parsing code to be multi-threaded, so I spent some time in attempting to make the business polling code multi-threaded.

Big mistake. Sort of.

Unlike Python, PowerShell doesn’t have any native support for .NET threading. I’m guessing that this is so because much of the underlying language primitives and/or the runspace within which PowerShell session state is saved is not thread-safe nor were they designed to be consumed as raw .NET threads to begin with. In its place, PowerShell uses “Jobs,” which are essentially scriptblocks or scripts executed within forked PowerShell sessions whose results are streamed back to their parent PowerShell session. While they’re better than having nothing at all, they are quite limiting in a few ways:

– PowerShell sessions usually consume about 20MB of RAM (or more depending on your $PROFILE and Modules directory), whereas .NET threads take up about 1MB.
– Every PowerShell session loads your $PROFILE, which can be highly, highly wasteful depending on its size and can delay initialization.
– Scripblocks sent to every child session runs within isolated runspaces, so sharing objects or state from the parent session is impossile without some modification.

(I know that PowerShell 4.0+ supports workflows, but they are also based on the Jobs engine and have the same set of problems and a different set of quirks, and I know how to work with Jobs better.

Knowing these limitations, I decided on spending some time creating a custom job queue implementation for Powershell that:

– Creates the concept of job pools by creating jobs that share common identifiers,
– Throttles the addition of jobs into job pools automatically when they go above a certain threshold,
– Reloads dependencies in the runspaces for every job,
– Serializes function calls and their arguments so that the consumer of the pool doesn’t have to, and
– Outputs errors and results from jobs within the pool as a pscustomobject for easy manipulation.

It took >10 infuriating hours for me to get this working right, but I eventually got it…and found it to be WORSE than what I had before for a few reasons:

– My AWS instance is very memory-constrained (only has 1GB of RAM), so when taking OS overhead into account, I only needed a few concurrent PowerShell sessions to send my machine into swap city (10 powershell sessions = ~200 threads = 200 MB).
– PowerShell jobs don’t have access to the parent host console because of the whole isolated runspace thing, so there’s no way for me to easily debug issues within threads when they come up until AFTER they’re done executing. This could be problematic if issues arise while retrieving business data from Google sincetheir API request quota is really low for server-side requests (2000 requests/day).
– God it was so fucking slow.

What was funny was that I wasn’t that upset about this. I was much more upset whenever I ran into a bug and thought that I’d have to spend my entire weekend on this. Now that I know that this approach sucks (for now; I think it’ll be much more useful when I move this onto a real programming language), because I still have the original code that does this serially, I can spend today focusing on the real Goliath: event discovery within Twitter.

Onwards!

Advertisements
Day 33: Choose your languages wisely.

Day 30: I’m scared…and that’s okay.

STATUS: Fucking OAuth.

MOOD: I’m so close!

I’m scared of a bigger company (or person smarter than me) releasing my product faster than I can and taking the market with them.

I’m scared of encountering a huge show-stopper months into development.

I’m scared of never getting any funding. (I’m trying to get enough funding to focus on this full-time. I have an idea of the expenses involved for the business, but running/developing/networking it on top of a full-time job is starting to get difficult.)

I’m scared of being unable to convince people that working with me is worth their time.

I’m scared of failure and having to restart.

I’m scared of never seeing success.

I’m scared of giving up.

What’s different about these fears from, say, my fear of riding those 400-foot free-fall rides is that this is *good* fear. The fear doesn’t immobilize me. The fear ignites me.

I still feel incredibly alive after a multi-hour coding session or coffee/pitch meeting. The implicit rejections I get (such as my landing page and social networks being mostly dormant, or that I’m still far away from an interactive demo of some kind) aren’t making me afraid of continuing; they make me think about how I can do better and push forward. Every line of code I write brings me closer to putting this out in the world, and that’s enough to silence my fears for a short while.

It has been said that there are few things in life that make a person more vulnerable than starting a business. To that, I say this: if this shit were easy, everyone would do it, and I’d still be chasing the hard stuff anyway. I’ll take a little bit of hard work for a lot of awesome later down the line over stability and safety any day of the week!

Day 30: I’m scared…and that’s okay.

Day 27: Email ain’t easy, bro.

STATUS: Spent 12+ hours trying to migrate my personal email into Office 365 from gmail. Spent 0+ hours on thinglistr. Coffee maker didn’t get delivered. Fucking FedEx.

MOOD: Angry. Nothing fucking works.

————-

So I decided to stay home today to wait on the coffee maker. I usually use Amazon Lockers for this, but the one nearby me was broken, so I took the chance.

I figured that this would (finally) be a good time to move off of gmail (again) and onto a service that’s more cross-platform. Surprisingly, Microsoft’s Outlook.com service seemed to fit the bill.

Here’s where 12+ of my hours went:

1) Setting up Outlook.com online, only to find out that it STILL only supports EAS on mobiles. Desktops have to use IMAP. No CalDAV or CardDAV either, so good luck getting your contacts and calendar on your mac.

2) Trying to share my Office 365 subscription with my girlfriend. Got my account locked out a few times.

3) My account got locked out permanently. Awesome. (Just spent a few hours migrating messages from a few gmail accounts onto it.)

4) Trying to port my phone number to my cellular plan from Google Voice didn’t happen right away as expected, and calls were dropping for some people but not others.

5) Decided that maybe getting Office 365 for Business was a better move (yay, personal domain name!). Forgot to disable two-factor auth on my domain management system, so need to get another one while that gets unlocked.

6) Setting this up was actually much smoother than expected….until I had to do the migration. (This is what I’m doing now.) No tools out there to make it easy; you’ve gotta move those mailboxes yourself.

7) Mail.app REALLY sucks…but it’s pretty and simple.

I’m writing today off.

Day 27: Email ain’t easy, bro.

Day 22: My mockups are so fire, I’M SIGNING MYSELF UP FOR THE WAIT LIST

STATUS: I spent a few hours putting together a mockup that actually doesn’t look like shit!

MOOD: HYPE

————————————

I’m tired and don’t have much to say (I sleep 4-5 hours now on the weekdays; I used to get 9. I’ll sleep WHEN I’M RICH(I just need five dollars)), but I spent a few hours putting these together and am DAMNED PROUD OF THEM.

Someone please deflate my pride and tell me why they suck terribly and that I should stick to my heralded drawing of stick figures (I actually can’t draw straight lines that well) and writing code…or something like that.

thinglistr-main

Figure 1: thinglistr on the desktop

thinglistr-mobile-main

Figure 2: thinglistr on mobile

Day 22: My mockups are so fire, I’M SIGNING MYSELF UP FOR THE WAIT LIST

Day 18: A new approach.

STATUS: I’m going to take a break from building thinglistr and, instead, focus on getting its story solid.

MOOD: I’m surprised I didn’t pass out while writing this.

————

I finished Startup Weekend NYC a few hours ago. Our group didn’t place, but we did get an “honorable mention” by one of the judges. I didn’t actually show the product well enough, and even though the pitch itself was pretty good, other groups had much more to offer in that respect. I agree with their decision.

Regardless of how we fared, the connections and friendships I made through that ~30 hour grinder were the real prize. I met a whole collective of really fucking smart and ambitious people that I would have never met in any other situation. I might (might) even have a team forming for the project that we pitched. That’s a fucking win in my book.

Despite sleeping maybe nine hours between these three days, I felt drunk with energy the entire time. We had high points and low points, but never low enough to cause in-team drama. Our team dynamics were amazing. I think it’s rare to assemble a team that works together so well that easily, so that meant a lot to me. I did everything I could to ensure that the team dynamic did not fray, even if it meant reliquishing control and having my core concept get vetoed. Many people say that the team matters way more than the idea, and I can easily see why.

So what of thinglistr? Well, ironically enough, it took a weekend bootcamp in a sector that had nothing to do with my potential product to convince me that I’m doing my product wrong. (Another reason why I don’t care at all about having to pay $125 in credit; I might have kept making wrong decisions had I not gone.)

I spent too much time building and not enough time solidifying that my concept is something that people will actually use. I’ve asked a handful of people, which is fine…for a tool. My idea needs tens of thousands of users repeatedly using the site per day. A handful of people validating the idea will not work, and using that as a talking point with an investor or copartner will probably not get me very far.

This event also made me realize something very important: visuals matter WAY more than functionality. It’s really, really difficult to ace the MVP because “minimum” threshold differs between people. For me, I thought that the minimum I needed to have before pitching my idea was a working product that demonstrated thinglistr in action.

Well, the team that won this damn show didn’t write a single line of code. It was all mocked up. The reason why they won wasn’t because of their graphics; it was because of their approach.

If I can convince investors that (a) finding happy hours, bar specials and lower-key “events” is very much a real problem, (b) that several thousand people said the same and (c) that my solution will be difficult for the competition to replicate, I might have a chance.

I guess i do need to start thinking in terms of billions of dollars in revenue if I’m going to get serious about this. I’m guessing that that’s the only revenue point where these activities make sense.

Day 18: A new approach.