Pike-devel January 2010

pike-devel@lists.lysator.liu.se

5 participants
14 discussions

Tabular.pike 1.8
by Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) ＠ Pike (-) developers forum 05 Jan '10

05 Jan '10

+ if(String.width(out)>8) + out=string_to_utf8(out); Ok, seriously, I think for every proper use of String.width (are there any?) there are at least two improper ones. If the string is supposed to be transport encoded with UTF-8, you need to do it for _all_ strings, otherwise you will get invalid UTF-8 if the string contains characters > 127 but < 256. But who says that a "tabular" should be UTF-8 encoded anyway? For a format that has no builtin encoding indicator, transport en/decoding should be the task of the caller.

4 9

convert_1_to_0() truncating instead of skipping?
by Stephen R. van den Berg 03 Jan '10

03 Jan '10

I noticed that e.g. convert_1_to_0() (actually any conversion that goes from wider to narrower) in stralloc.c just chops off the wider part instead of actually leaving out the characters which cannot be represented in the narrower string. Is there are a reason why it is truncated instead of skipped? I'd presume that actually omitting the characters which cannot be represented would be more correct. Simply truncating them generates garbage. And, subsequently, if everyone agrees to the above, can/should I fix this? -- Sincerely, Stephen R. van den Berg. "If you make people think they're thinking, they'll love you; but if you really make them think, they'll hate you."

2 2

String.strim_whites(), any objections of other suggestions?
by Stephen R. van den Berg 03 Jan '10

03 Jan '10

I found myself reimplementing this functionality in a number of projects over the course of the past ten years, and I finally bit the bullet. Any objections when this is committed? Or perhaps better suggestions as to the naming and/or implementation? commit dda53cbadcf125f207878e8a5ac04ac936b21782 Author: Stephen R. van den Berg <srb(a)cuci.nl> Date: Fri Jan 1 23:12:18 2010 +0100 New member String.strim_whites which sanitises a string by stripping leading and trailing whitespace, in addition to reducing all strains of spaces and tabs to a single space. diff --git a/lib/modules/String.pmod/module.pmod b/lib/modules/String.pmod/module.pmod index 2696d93..e70c848 100644 --- a/lib/modules/String.pmod/module.pmod +++ b/lib/modules/String.pmod/module.pmod @@ -6,6 +6,7 @@ constant Buffer = __builtin.Buffer; constant count=__builtin.string_count; constant width=__builtin.string_width; constant trim_whites = __builtin.string_trim_whites; +constant strim_whites = __builtin.string_strim_whites; constant trim_all_whites = __builtin.string_trim_all_whites; constant Iterator = __builtin.string_iterator; constant SplitIterator = __builtin.string_split_iterator; diff --git a/src/builtin.cmod b/src/builtin.cmod index 4085f4e..1dda4dd 100644 --- a/src/builtin.cmod +++ b/src/builtin.cmod @@ -750,6 +750,55 @@ PIKEFUN string string_trim_whites (string s) RETURN string_slice (s, start, end + 1 - start); } +/*! @decl string strim_whites (string s) + *! @belongs String + *! + *! Trim leading and trailing spaces and tabs from the string @[s]. + *! Also reduces strains of multiple spaces and tabs to a single space. + */ +PMOD_EXPORT +PIKEFUN string string_strim_whites (string s) + errname String.strim_whites; + optflags OPT_TRY_OPTIMIZE; +{ size_t len = s->len; + void *src = s->str; + unsigned shift = s->size_shift; + unsigned foundspace = 0; + struct string_builder sb; + init_string_builder_alloc (&sb, len, shift); + sb.known_shift = shift; + + switch (shift) { +#define DO_IT(TYPE) \ + { TYPE *start = src, *end = start+len, *dst = (void*)sb.s->str; \ + for (; start < end; start++) { \ + int chr = *start; \ + if (chr != ' ' && chr != '\t') break; \ + } \ + for (; start < end; start++) { \ + int chr = *start; \ + if (chr == ' ' || chr == '\t') \ + if (foundspace) \ + continue; \ + else \ + chr=' ',foundspace=1; \ + else \ + foundspace=0; \ + *dst++ = chr; \ + } \ + len = dst - (TYPE*)sb.s->str; \ + } + case 0: DO_IT (p_wchar0); break; + case 1: DO_IT (p_wchar1); break; + case 2: DO_IT (p_wchar2); break; +#undef DO_IT + } + if (foundspace) + len--; + sb.s->len = len; + RETURN finish_string_builder (&sb); +} + /*! @decl string trim_all_whites (string s) *! @belongs String *! diff --git a/src/builtin_functions.h b/src/builtin_functions.h index 9da458a..1183d21 100644 --- a/src/builtin_functions.h +++ b/src/builtin_functions.h @@ -181,6 +181,7 @@ PMOD_EXPORT void f_ctime(INT32 args); PMOD_EXPORT void f_mkmapping(INT32 args); PMOD_EXPORT void f_string_count(INT32 args); PMOD_EXPORT void f_string_trim_whites(INT32 args); +PMOD_EXPORT void f_string_strim_whites(INT32 args); PMOD_EXPORT void f_string_trim_all_whites(INT32 args); PMOD_EXPORT void f_program_implements(INT32 args); PMOD_EXPORT void f_program_inherits(INT32 args); -- Sincerely, Stephen R. van den Berg.

3 2

Re: String.strim_whites(), any objections of other suggestions?
by Stephen R. van den Berg 02 Jan '10

02 Jan '10

Peter Bortas @ Pike developers forum wrote: >A more descriptive name would be good. The trouble with "more descriptive" is that it implies absolute accuracy in the naming. I.e. the following names crossed my mind: - sanitise_whites(): The definition of "to sanitise" is subject to personal taste, and therefore inaccurate. - strip_and_trim_whites(): Some whites are stripped, but not all, and some tabs are being replaced by spaces, so this name is inadequate as well. - replace_multiple_with_one_and_trim_whites(): would be sort of accurate, but is ridiculously long. Considering the above, I thought that "strim_whites()" is not so bad, since it somehow contracts "strip and trim" to "strim", which means it doesn't say "strip" explicitly (since it doesn't strip everything), and it does more than trimming alone. And since the name is too short to exactly describe what happens, one needs to look it up in the docs to find out what it does exactly anyway. AFAIK there is no equivalent function in other languages, so borrowing the name from there was/is not possible (maybe someone can prove me wrong here?). -- Sincerely, Stephen R. van den Berg. "The difficult we do today; the impossible takes a little longer."

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel January 2010