Once, while investigating something, I deliberately disabled my Internet connection.
Then, I noticed one of Hugo’s tests failing. Obviously it accesses the actual Internet.
Located in hugolib/embedded_shortcodes_test.go, its name is TestShortcodeTweet (last updated June 12, 2016). It accesses Twitter.com to retrieve a certain tweet, and verifies its contents match a string stored directly in the test.
This is the sole test (among all of Hugo’s) which accesses the actual Internet (outside of one’s local machine).
It’s conceivable that other tests—skipped on my Windows system—also access the Internet. They’re skipped because I haven’t installed certain optional Hugo support programs: Asciidoc, Pygments, and rst2html.
So—should Hugo’s tests never access the actual Internet?
I found an article, How to Do Symmetric API Testing in Go, which reviews another article, Symmetric API Testing by Aditya Mukerjee, maintainer of Anaconda, the Twitter client library for Go.
As I understand, he advises people to run a utility program to capture Twitter’s responses (occasionally), which automatically would generate Go struct definitions to be stored in a file (in version control).
That seems too complicated for Hugo’s simple Twitter use case. Yet at least Hugo might store (in some file) the single tweet response it verifies. By that step, Hugo would advance somewhat (probably sufficiently) in the direction of Anaconda.
TestShortcodeTweet could use the content of that file to mock Twitter’s response.
Just as now, if (or when) Hugo’s Twitter shortcode stops working (due to a change in Twitter’s API or in Hugo code: whatever), we’d find out. Not as immediately as by live testing—yet presumably, users would let us know.
Or should we continue to access Twitter.com in each test run? Twitter seems fairly reliable.