In the CPAN::Digger project I've encountered an issue. The process cloning the repositories will ask for a "Username" and get stuck whenever an incorrect URL was supplied. This could happen by a plain typo, if the repository was made private, and maybe other cases.

If someone has renamed a repository and forgot to update the URL in the META files the clone should still work, at least for some time. This is what's happening to the old name of the CPAN::Digger repository.

All of these cases must be reported, but they should not cause the checking process to get stuck.

One of the fixes I have applied is checking if the repository URL returns "OK 200" when accessing with a simple GET requests.

This is a sample code showing 3 different ways how to do it and how to verify that code.

In this example we use LWP::UserAgent to send a GET request and then we check if the response was "200 OK" or "404 Not Found".

In the default behavior of the LWP::UserAgent it allows for redirects and reports the results of the last call. This is what you can see in the first solution.

In the second solution we set the max_redirect to be 0 and thus the response we got was "301 Moved Permanently".

The CPAN::Digger should probably report these cases so the author can notice them and fix them.

examples/check_git_url.t

use strict;
use warnings;

use Test::More;
use LWP::UserAgent;

is check('https://github.com/szabgab/CPAN-Digger'), '200 OK';
is check('https://github.com/szabgab/no-such-repo'), '404 Not Found'; # same for private repositories
is check('https://github.com/szabgab/cpan-digger-new'), '200 OK';


is check_no_redirect('https://github.com/szabgab/CPAN-Digger'), '200 OK';
is check_no_redirect('https://github.com/szabgab/no-such-repo'), '404 Not Found';
is check_no_redirect('https://github.com/szabgab/cpan-digger-new'), '301 Moved Permanently';


is_deeply check_redirect('https://github.com/szabgab/CPAN-Digger'),     [200, 0];
is_deeply check_redirect('https://github.com/szabgab/no-such-repo'),    [404, 0];
is_deeply check_redirect('https://github.com/szabgab/cpan-digger-new'), [200, 1];

done_testing;

sub check {
    my ($url) = @_;
    my $ua = LWP::UserAgent->new(timeout => 5);
    my $response = $ua->get($url);
    return $response->status_line;
}


sub check_no_redirect {
    my ($url) = @_;
    my $ua = LWP::UserAgent->new(timeout => 5, max_redirect => 0);
    my $response = $ua->get($url);
    return $response->status_line;
}

sub check_redirect {
    my ($url) = @_;
    my $ua = LWP::UserAgent->new(timeout => 5);
    my $response = $ua->get($url);
    return [$response->code, ($response->redirects ? 1 : 0)];
}

Remember, you pick one of the 3 functions, whichever feels more convenient to you. Or you roll your own.