Questions to ask cloud vendors

Part of our methodology is to make sure we ask enough questions to fill out the weighted matrix we use for scoring cloud vendors. When we schedule a vendor interview, we will send them the below questions, not so that they can answer and send them back, but so they can have the right people available on the phone to discuss them. Many times this can prompt other discussions to help us better understand how they operate which helps us better score them

Feel free to use the below in your own cloud scoring. Or if you are too busy and want us to do it, by all means let us know.

--------------------------
We are reviewing our cloud vendors and we would like to have a discussion to understand your company, processes, contracts etc. We are very pro-cloud so please do not be alarmed. We are not using this as a reason to disqualify you, but simply to understand what our risks are and if we need to take additional steps to mitigate these risks. In essence this is our due diligence.

You do not need to answer these questions. We think there is more value in having a discussion around these topucs to we can drill in where we need to.


Disaster Recovery and Business Continuity
Do you have redundant sites designed for auto-failover? How long does it take for the redundant site to take over. Does this include the time to decide to fail over?

What kind of RTO/RPO are in place and are they actually tested against?

Do you have geographic redundancy?

Do you perform backups. and how granular and far back can we recover? Can we recover accidentally deleted files?

What impact does a failed HD, server, cabinet, switch, data center have?

E-discovery
Is it possible to search for any files, records or emails by keyword, owner, date?

Can we do legal holds by user, file, keyword?

Can we get access to “access logs” in the event we need to? If so how far back can we get? What does it show us?

Can you see who your users are sharing with? If so can you easily remove access from an enterprise level?

Are there any additional costs for this?

Stability
Do you have a public, or better yet third party, site that shows real time status for transparent operations?

Is code/data in escrow? If so how often does it get updated?
Can we request a backup or better yet, just take a backup,  of our data including any customizations?

Company financials
Can you share financial data for the last few years?

Are you private or publicly owned?

Are you cash flow positive?

Are you adding new customers? How many?

Do you know your  NPS (Net Promoter Score)?

Who are the key investors and management team?

Authentication
Do you support automatic provisioning and de-provisioning of user accounts?

Do you support LDAP.RADIUS or even better SAML authentication and authorization back to us?

Do you use encryption? If so is it for data in flight (moving across the network), at rest (on disk or in memory) or both? What kind of encryption is it?

Compliance and Privacy
Do we get notified of an investigation?

Can our data be seized as part of another companies investigation?

Is data recoverable by your organization?

Contract
If I want to leave, for example if they get bought by a competitor, can I get my data out and go?

Are there financial penalties for service level agreement failure.

Are maximum increases baked in?

How much notice do you need to give us to terminate?

How much notice do we need to give them if we want to leave?  

Does the contract auto-renew? If so what are the terms?


Performance
Are you globally load balanced?

Do you use a specialized network or other Wan optimization, for better performance?.

Who do you use for WAN connectivity? I

Do you offer “offline” ability? If so it is automatic, or does the user need to know that they will be offline and plan accordingly?

How scalable is the system? Can we scale to 10X, 100X, 1000X? What is the breaking point? And how quickly can new capacity be added if needed to scale higher?

Development
Do you offer built in integration tools? If so to what other systems?

What toolset is used for  “custom development”?

Can we customize the login or "main" screens with our logo, colors, etc?  How much canned vendor wording needs to stay, or can we wipe it all clean if we want?

How vibrant is your customer community?  Please point us to the public discussion Q&A boards for your product.  What are the main Twitter hashtags your customers use when they talk about you?  Do you monitor them?

Do you gather product ideas publicly from your customers, where they can read and vote on each others' ideas?  

Are the API’s well documented? How often do they change?

How much of the system can be operated with API’s versus the user interface?

How is authentication handled in the APIs?

Support
Are you staffed 24/7?

Is it onsite, email, phone, web or all?

What sort of response time is available?  

When we open a case, do wetalk to someone knowledgeable right away, or do we have to have a number assigned and wait for someone to call us back?

What is the average tenure of the tier1 staff?

Is there a public knowledgebase available? Is it the same as the internal one or is it filtered? How good is the search or UI?

Can anyone from our company or IT staff call, or do we only get a certain amount of “authorized users”?

Can I use support as part of my evaluation period?

Does the same engineer own a case throughout its lifecycle, or are cases "handed off" among tiers?

Does your support staff prefer or default to email, phone, or web?

What's your support team's onshore / offshore and employee / contractor breakdown?

Does your support team also support your own internal implementation of your software?

Can we log tickets on a portal and carry on the whole conversation there?


Architecture
How quickly do new features show up? Do we need to do anything or do we “magically” get them? How much notice do you give us for training users?

Are you built on a multitenant system?

Do you support multiple clients, like iphone, android, blackberry as well as tablets? Are you strictly HTML5/browser based? If so which browsers?

How long does it take to bring a development or evaluation system online? Does it need human intervention?

Are there any special restrictions on sandbox or developer systems? (storage,  API’s, queries, users etc)

Do we have the ability to give customers "sandbox" instances, i.e. exact copies (including metadata and data) of our production instance, for testing purposes?  If so, can we create them directly, or does it take a request and lead time?  How often can sandboxes be refreshed?

To what extent is the system "metadata-driven," i.e. how much can I change without code?

User Experience
How accurate is the development roadmap for the next release? Next year’s worth of releases?

Are there mobile versions of the application? If so what limitations, if any, are there?

Is user training available online?

How extensive is the help system?

Administration
Can my system administrators "log in as" other users, in order to diagnose problems or confirm security settings?

For an implementation our size, how much IT time is typically required to manage the system?

How detailed are users' activities logged?  What can a sysadmin see?  e.g. can we see who logged in when; what functions they used; what pages they viewed; etc.