If I set your service to monitor a webpage under Cloudflare Always Online caching, will it detect downtime?

If Cloudflare returns an HTTP error code, we'll consider it as down otherwise it'll be considered as up. The server IP address can be also monitored directly.