We often get asked how much parallel computing power each user on our platform gets. The short answer is that it’s variable and left up to our algorithms by default, but if you need, it’s easy to specify that you want exactly N cores using our Realtime Cores feature—a couple of clicks on our web dashboard, or one line of code.
Standard Service (w/o Realtime)
By default, your jobs take advantage of cores that are shared by all of our users. We do not have a set number of total cores in our system, and there are no set number of cores per user. Instead, PiCloud continually and automatically estimates the workload we have and will have in the next hour. There are two components to this estimation:
- Periodic Jobs: A large class of users runs jobs periodically, whether it’s every minute, hour, or day. Over time our system learns the amount of load each of these users contributes, and increases server capacity in anticipation of periodic jobs.
- Aperiodic Jobs: For users who do not have a predictable pattern, we scale our worker nodes once their jobs have been added to our queue. The number of workers we bring up depends on how long we estimate their jobs will take.
How do we estimate the runtime of a job?
We estimate the runtime of enqueued jobs by the average and variance of runtimes of similar completed jobs; a similar job is any job by the user that executed the same Python function with the same PiCloud-specific reserved keyword arguments. Keyword arguments such as
_type are important since they could affect the speed at which a job is processed.
How does your workload affect the number of workers we bring up?
Since we use Amazon Web Service’s servers, we are charged for every hour we have a server up. To manage our costs, we try to make sure our servers spend as much time as possible processing jobs, rather than sitting idle. Here’s a rule of thumb for how this affects you: If each of your jobs takes an hour to process, PiCloud will automatically scale such that all of your jobs are running simultaneously in parallel. We’re happy to bring up as many as you need since you’re keeping our servers busy for the full hour increment that we rent from Amazon. If each of your jobs takes a half hour, we’ll run about half of your jobs in parallel at a time, so that each core will be busy for the full hour running two jobs each. In practice, our scaling algorithm is more liberal than this, but this should give you a good conservative ballpark.
How much compute power should you expect?
If you’re just running a small batch of short jobs, you’ll generally get less than 10 cores. If you’re running a large number of long-running jobs, then using the above algorithm, we could potentially bring up hundreds of cores for you.
The model of our standard service is ideal for users trying out PiCloud for the first time, users with relaxed response-time requirements, and users with large batch jobs. Our realtime service is for our other class of users who need N cores to be ready at a moment’s notice, even if that means those cores will be sitting idle at times.
How it Works
You tell us the type of core you want, and the number you want, and we’ll bring them online just for you. Your jobs will be placed in both your own realtime queue, as well as our general queue. So if you have reserved 80 cores, and you check in 200 jobs, at least 80 jobs will be immediately be dequeued and begin processing on your realtime cores. Potentially more than 80 may run simultaneously since you still have access to the cores available in our standard service.
Right now, we charge a fixed cost for every core hour (rounded up). If you reserve realtime cores, and you run no jobs, you’ll be charged that amount. If you run jobs, you’ll be charged both the cost of the realtime cores, as well as the cost of our standard service for the number of compute hours your jobs run for. For example, if you reserve 10 c1 cores for an hour, and run 1 compute hour worth of jobs on c1 cores during that time, you’ll pay $0.15 + $0.05 = $0.20 cents total (See our pricing page). The problem we have with this is two fold:
- The fixed cost of each realtime reservation needs to be sufficiently high to cover our expenses in the event a user does not run any jobs. Charging our elastic rate on top of that is unfairly expensive.
- If our standard service scaled a user to 100 cores, and the user then reserved 30 cores, they would not see any additional compute power, while paying substantially more. While it’s true that they would be guaranteed that compute power without any risk of crowding from other users, we don’t believe users were getting the best bang for their buck.
We love users who use realtime because they offer our automatically-scaling system a level of predictability; we don’t want to dis-incentivize this behavior. So we’re going to change our policy as follows: Each realtime core, rather than being an hourly charge, will instead be an hourly minimum. Hypothetically, if you reserved a realtime core that has a minimum of $0.10/hour, then after an hour you would be charged the greater of $0.10 or the bill you accrued from using our standard service. If you don’t run any jobs, that’s $0.10. If you run some jobs, but the standard service bill doesn’t exceed $0.10, then you pay $0.10. If you run enough jobs to exceed $0.10, say $0.15, then you pay just $0.15. We’ll let you all know when we’ve made this change to our pricing!
How to Reserve Realtime Cores
Go to the realtime tab on our web dashboard.
Select the type of core, and the number of cores you want for us to keep available for you, and then hit submit.
When the cores are ready, you’ll receive an email and the “start time” will be set.
When you run jobs, make sure that you set the
_type keyword to the core type that you have reserved. In this example, we reserved 160 c2 cores.
Alternatively, reserving realtime cores will be possible through the Python console using the
cloud library in its next release:
How long does it take to allocate Realtime Cores?
We’ve been working hard making allocation time faster and more consistent, which has been a common issue raised by users. With our latest design, it should take no more than 15 minutes for cores to be provisioned!
How many cores can you allocate?
We have users who provision hundreds and thousands of cores using our realtime service. If you’re worried you might break our backs, file a support ticket and let us know what you’re up to.
The takeaway we want to leave with you is as follows: The PiCloud platform will not only get you up and running in no time with no configuration, but will also be able to meet your exact computational requirements if necessary—even if you need 1000 cores in 15 minutes. It’s not a question of if, but how, and we’re working to make how as easy as possible.