Wget HEAD request?

Question

I'd like to send the HTTP HEAD request using wget. Is it possible?

score 67 · Accepted Answer · answered Oct 09 '10 at 19:14

67

It's not wget, but you can do that quite easily by using curl.

curl -I http://www.superuser.com/

Produces this output:

HTTP/1.1 301 Moved Permanently                        
Content-Length: 144                       
Content-Type: text/html; charset=UTF-8     
Location: http://superuser.com/
Date: Sat, 09 Oct 2010 19:11:50 GMT

answered Oct 09 '10 at 19:14

SleighBoy

2,240
14
16

This is exactly what I want. – Lenik Oct 11 '10 at 15:23
3

`-I` is equivalent to `--head`. – Nicolas Dec 10 '15 at 20:48
2

If you need self-signed certificate based `https`, you can also add `-k` or `--insecure` – Mike Aski Feb 06 '19 at 17:31
Given that the question explicitly mentioned `wget` i can't understand why an answer using `curl` can be considered the best, especially that there may be contexts where just one of the two is available. – Treviño Feb 24 '21 at 22:39
unfortunately, `curl -I` does not give the URL to where it is redirected when it's a 301 or 302 – Cyril Chaboisseau Sep 02 '22 at 09:29

score 46 · Answer 2 · edited Sep 28 '12 at 09:08

46

Try:

wget -S --spider www.example.com

You can also pass -O /dev/null to prevent wget from writing HTTP response to a file.

edited Sep 28 '12 at 09:08

user68278

103
3

answered Oct 07 '10 at 17:10

Casual Coder

3,972
27
15

4

`-S` will *show* headers, but it executes a `GET`, not a `HEAD`. In other words, it will fetch the entire URL. – Dan Dascalescu Feb 28 '14 at 23:21
17

`wget -S --spider http://localhost` log created in apache server is `127.0.0.1 - - [04/Mar/2014:15:36:32 +0100] "HEAD / HTTP/1.1" 200 314 "-" "Wget/1.13.4 (linux-gnu)"` – Casual Coder Mar 04 '14 at 14:38
2

from the manual : --spider When invoked with this option, Wget will behave as a Web spider which means that it will not download the pages, just check that they are there. – ychaouche Apr 06 '20 at 19:37
It is still creating an empty directory structure, so I found that `-nd` (`-nH --cut-dirs=100` also works) works to suppress that. – haridsv Sep 16 '20 at 14:38
`-nv` or `--no-verbose` for just the facts. – Bob Stein Jun 22 '21 at 16:27
You can also try the example on a very large remote file, it completes instantly, so it doesn't download it. – adrianTNT Mar 13 '22 at 11:36

score 23 · Answer 3 · edited Jan 13 '17 at 16:08

There isn't any need for curl.

With Wget, adding --spider implies that you want to send a HEAD request (as opposed to GET or POST).

This is a great minimalistic way of checking if a URL responds or not. You can for example use this in scripted checks, and the HEAD operation will make sure you do not put any load on neither the network nor the target webserver.

Bonus information: If Wget gets an HTTP error 500 from the server when it performs the HEAD it will then move on to perform a GET against the same URL. I don't know the reasoning for this design. This is the reason why you may see both a HEAD and a GET request being performed against the server. If nothing is wrong then only a HEAD request is performed. You can disable this functionality with the --tries option to limit Wget to only one attempt.

All in all, I recommend this for testing if an URL is responding:

# This works in Bash and derivatives
wget_output=$(wget --spider --tries 1 $URL  2>&1)
wget_exit_code=$?

if [ $wget_exit_code -ne 0 ]; then
    # Something went wrong
    echo "$URL is not responding"
    echo "Output from wget: "
    echo "$wget_output"
else
    echo "Check succeeded: $URL is responding"
fi

score 4 · Answer 4 · edited Nov 08 '16 at 11:06

4

wget -S gets file:

Content-Length: 2316, Length: 2316 (2.3K) [text/plain], Saving to: `index.html'

wget --spider gets headers:

Spider mode enabled. Check if remote file exists., Length: unspecified [text/plain] Remote file exists.

edited Nov 08 '16 at 11:06

7ochem

161
2
11

answered Apr 05 '14 at 12:53

LanPartacz

41
1

score 2 · Answer 5 · answered Jun 30 '20 at 05:08

2

Ubuntu 19.10 Manpage: Wget

   --method=HTTP-Method
       For the purpose of RESTful scripting, Wget allows sending of other HTTP Methods
       without the need to explicitly set them using --header=Header-Line.  Wget will use
       whatever string is passed to it after --method as the HTTP Method to the server.

Used the following in my bash script and can confirm it works as expected!

wget --method=HEAD https://www.website.com/

answered Jun 30 '20 at 05:08

phbits

121
2

1

At least the Wget version in Alpine Linux v3.12 does not support the `--method` parameter. – Gogowitsch Jan 23 '21 at 21:12

score 1 · Answer 6 · answered Oct 07 '10 at 16:09

1

Though not wget, many perl installs with lwp module will have a HEAD command installed.

answered Oct 07 '10 at 16:09

Rich Homolka

31,057
6
55
80

Wget HEAD request?

6 Answers6

Linked