Pike-devel August 2014

pike-devel@lists.lysator.liu.se

9 participants
29 discussions

Re: Codepage decoding tables have incorrect data
by Chris Angelico 07 Aug '14

07 Aug '14

On Fri, Aug 8, 2014 at 1:50 AM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > Ok. Then my approach would be to keep things as they are until > someone comes up with a specific complaint. :-) As I said, the box > drawing stuff is fixed, so feel free to raise a new issue if you find > something else which is a concrete problem. Yeah, I agree. Codepages are a mess anyway; as long as there's some kind of reasonably sane translation done, it's as good as it's going to be. > (As a side note, I think you were justified in assuming thick/double > lines was a font issue. There is no semantical difference, so I > imagine the only reason why Unicode assigns separate codepoints to > them instead of treating them as font variants is for compatibility > with some other encoding that has both...) No idea. To be quite honest, I wasn't even aware of those other box-drawing characters' existence until I came across this... there's so much Unicode that it's impossible to see it all. ChrisA

1 0

Re: Codepage decoding tables have incorrect data
by Chris Angelico 07 Aug '14

07 Aug '14

On Fri, Aug 8, 2014 at 1:30 AM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > I have pushed fixes for the box drawing characters, and for the > incorrect forms of arabic characters. This includes fixes to IBM 851 > and IBM 868, which were not part of your patch. > > Many of the remaining changes suggested are a bit dubious though. > > For example, in IBM 437 you proposed the following changes: > > 0xe1: GREEK SMALL LETTER BETA -> LATIN SMALL LETTER SHARP S > 0xe6: GREEK SMALL LETTER MU -> MICRO SIGN > 0xed: EMPTY SET -> GREEK SMALL LETTER PHI Like I said, I didn't have an authoritative source (as I didn't know where to find one), and so I used two non-authoritative sources, which happened to agree: the unicode.org files, and Python 3. Up to you what you want to apply; the part that I was most noticing was the box drawing characters, which I thought was actually a font error (as I wasn't seeing the double lines). ChrisA

2 1

Re: Codepage decoding tables have incorrect data
by Chris Angelico 04 Aug '14

04 Aug '14

On Tue, Aug 5, 2014 at 12:20 AM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: >>http://www.ietf.org/mail-archive/web/ietf/current/msg47378.html >>http://www.ietf.org/mail-archive/web/ietf/current/msg47387.html >>http://www.ietf.org/mail-archive/web/ietf/current/msg47444.html >>http://www.ietf.org/mail-archive/web/ietf/current/msg47574.html > > These posts appear to be about the _names_ assigned to characters in > RFC 1345, which isn't relevant here. That's true, in the specifics being dealt with at the time, but the general comments that 1345 isn't a standard and has inaccuracies are true of it all. I would still be inclined to follow the unicode.org files rather than RFC 1345; if nothing else, for consistency with other tools. ChrisA

2 1

Missing files in export
by Magnus Holmgren, Millnet/Lysator/Debian/Mensa ＠ Pike developers forum 04 Aug '14

04 Aug '14

I believe src/export_list is missing the following files, which install.pike tries to install: /refdoc/doxfilter.sh /refdoc/doxygen.cfg

1 0

Re: Codepage decoding tables have incorrect data
by Chris Angelico 04 Aug '14

04 Aug '14

On Mon, Aug 4, 2014 at 5:55 PM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > I haven't checked whether the changes to the codepages that you > suggest make sense yet (the current tables are generated from > RFC1345, so if there is a discrepancy it should be investigated more > closely), but I don't really see any point in adding your script to > the repository... Yeah, that's why I kept that separate. Figured it quite possibly wouldn't be useful :) Poking around on the internet showed up other places where RFC 1345 is discussed due to incompatibilities with current Unicode standards, such as this thread (highlights only): http://www.ietf.org/mail-archive/web/ietf/current/msg47378.html http://www.ietf.org/mail-archive/web/ietf/current/msg47387.html http://www.ietf.org/mail-archive/web/ietf/current/msg47444.html http://www.ietf.org/mail-archive/web/ietf/current/msg47574.html It seems that, in the event of a conflict, the Unicode files should be used rather than RFC 1345, which has been reported as having errors. ChrisA

2 1

Outdated GNU license texts.
by Magnus Holmgren, Millnet/Lysator/Debian/Mensa ＠ Pike developers forum 04 Aug '14

04 Aug '14

I just noticed that lib/modules/Tools.pmod/Legal.pmod/License.pmod/gpl.txt and lib/modules/Tools.pmod/Legal.pmod/License.pmod/lgpl.txt are very old; you should get the current ones from https://www.gnu.org/licenses/gpl-2.0.txt and https://www.gnu.org/licenses/lgpl-2.1.txt. Specifically, the old files in the repo contain an old postal address and gpl.txt refers to LGPL as the "GNU Library General Public License".

3 3

Codepage decoding tables have incorrect data
by Chris Angelico 04 Aug '14

04 Aug '14

Was doing some codepage work (playing with CP437), and found that the Pike CP-437 decoder actually produces wrong output, according to the unicode.org template file. I'm not sure what's going on here; is there some alternate standard that it's adhering to? Here are two scripts to compare codepage decoding in Pike and Python: int main(int argc,array(string) argv) { object decoder=Charset.decoder(argv[1]); for (int byte=128;byte<255;++byte) write("%02x: %04x\n",byte,decoder->feed((string)({byte}))->drain()[0]); } import sys for byte in range(128,255): print("%02x: %04x"%(byte,ord(bytes([byte]).decode(sys.argv[1],errors="replace")))) $ diff <(pike charsets.pike 437) <(python3 charsets.py 437) This should in theory be absolutely silent, but in practice it reports a number of differences in the decode. Wikipedia cites a file on www.unicode.org as its source: http://en.wikipedia.org/wiki/Code_page_437 http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/PC/CP437.TXT and that file agrees with Python's decoding. I've put together a script to download a file from the above URL and patch the info into the appropriate place in src/modules/_Charset/misc.c, and have run it on everything in the MICSFT/PC/ directory; the resulting patch is attached, as is a patch to create that file. ChrisA

2 2

Ancient touch syntax
by Magnus Holmgren, Millnet/Lysator/Debian/Mensa ＠ Pike developers forum 03 Aug '14

03 Aug '14

You use an obsolete touch syntax in src/Makefile.in. Current GNU touch apparently won't recognize it unless _POSIX2_VERSION is set to 199209. (For some reason, building 7.8.866 from exported source still works, but when I tried building 8.0.2 the same way (I used make snapshot_export), the rebuilding of interpret_functions.h failed since precompile.sh couldn't find pike, which shouldn't be surprising since it's not built yet at that point, is it?) Do you know any relevant systems where the following, which is POSIX.1-2001, wouldn't work? --- a/src/Makefile.in +++ b/src/Makefile.in @@ -1357,7 +1357,7 @@ precompile.sh: $(SRCDIR)/precompile.sh.i # recursive use of make. precompile.sh-stamp: @if [ -f precompile.sh ]; then :; else $(MAKE) precompile.sh; fi - @touch 0101010185 precompile.sh-stamp 2>/dev/null + @touch -t 198501010101 precompile.sh-stamp 2>/dev/null $(SRCDIR)/machine.h.in: $(SRCDIR)/stamp-h.in @if test -f $(SRCDIR)/machine.h.in; then :; else \

1 0

Redefinition of size_t
by Magnus Holmgren, Millnet/Lysator/Debian/Mensa ＠ Pike developers forum 03 Aug '14

03 Aug '14

This should work? --- Pike-v7.8.866.orig/src/modules/_Image_JPEG/transupp.c +++ Pike-v7.8.866/src/modules/_Image_JPEG/transupp.c @@ -36,17 +36,7 @@ * of internals, and added inclusion of jerror.h */ -#define FILE void -#define size_t unsigned int -/* NOTE: INT32 and INT16 are redefined by <jmorecfg.h>. */ -#if 0 -#ifdef INT16 -#undef INT16 -#endif /* INT16 */ -#ifdef INT32 -#undef INT32 -#endif -#endif /* 0 */ +#include <stdio.h> #define XMD_H /* Avoid INT16 / INT32 being redefined */

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel August 2014