Re: [Boost-Users] Re: Preprocessor library question

19 Dec 2002

      Thanks for your response, I'll look through it and try to get a full grasp
on everything you said.

Basically, my problem is this

I have to read in a file of fixed length records, manipulate them (e.g.,
sort, dedup, group, some custom processing ) and then write them out.  There
are about 40M records, so I want it to be as fast as possible.  If I know
the length of the record fields ahead of time, it is MUCH more efficient.
However, I would like the field lengths to be customizable at runtime...I
thought if I could precompute the program for various values of the 3
fields, then I could have the best of both worlds.  However, I forgot about
all the template instantiations...perhaps they'll just have to recompile
when they want a new field length.

Tanton
----- Original Message -----
From: "Paul Mensonides" 
To: 
Sent: Thursday, December 19, 2002 1:56 AM
Subject: [Boost-Users] Re: Preprocessor library question
...
"Tanton Gibbs"  wrote in message
news:024e01c2a725$6814d720$b048cc18@Corp.Acxiom.net...
...
I'm trying to create a template class based on user input.  For example:
template< int i, int j, int k > class T {};
int main() {
  int i, j, k;
  cin >> i >> j >> k;
  T t;
}
Naturally, this is not possible, as i, j, and k must be constants.
Therefore, I would like to limit the maximum value of i, j, and k to 100
...
create all permutations for T at compile time.  Basically,
something
like this:
Hi Tanton,
You realize that this will give you a 101*101*101 permutation right?
While
the pp-lib can generate it, that is a pretty big block of code to be
inside
a single function.  I would try to separate it into separate functions if
possible:
template struct bound {
    static void target() {
        T t;
        // ...
    }
};
typedef void (* pf)();
template pf bind(int k) {
    switch (k) {
        case 0:
            return &bound::target;
            break;
        case 1:
            return &bound::target;
            break;
// ...
default:
            return 0;
    }
}
template<int i> pf bind(int j, int k) {
    switch (j) {
        case 0:
            return bind(k);
        case 1:
            return bind(k);
// ...
default:
            return 0;
    }
}
pf bind(int i, int j, int k) {
    switch (i) {
        case 0:
            return bind<0>(j, k);
        case 1:
            return bind<1>(j, k);
// ...
default:
            return 0;
    }
}
int main() {
    int i, j, k;
    std::cin >> i >> j >> k;
    bind(i, j, k)();
    // ...
}
Will something like that work for you?  Something similar to this is *way*
better than having an if structure with 101*101*101 conditional tests.  If
so, here is an implementation:
#include <iostream>
#include 
#define LIMIT 100
template struct bound {
    static void target() {
        std::cout << i << ' ' << j << ' ' << k << &std::endl;
        return;
    }
};
void overflow() {
    std::cout << "overflow" << &std::endl;
    return;
}
typedef void (* pbound_f)();
template pbound_f bind(int k) {
    switch (k) {
        #define BOOST_PP_LOCAL_MACRO(k) \
            case k: \
                return &bound::target; \
            /**/
        #define BOOST_PP_LOCAL_LIMITS (0, LIMIT)
        #include BOOST_PP_LOCAL_ITERATE()
        default:
            return &overflow;
    }
}
template<int i> pbound_f bind(int j, int k) {
    switch (j) {
        #define BOOST_PP_LOCAL_MACRO(j) \
            case j: \
                return bind(k); \
            /**/
        #define BOOST_PP_LOCAL_LIMITS (0, LIMIT)
        #include BOOST_PP_LOCAL_ITERATE()
        default:
            return &overflow;
    }
}
pbound_f bind(int i, int j, int k) {
    switch (i) {
        #define BOOST_PP_LOCAL_MACRO(i) \
            case i: \
                return bind<i>(j, k); \
            /**/
        #define BOOST_PP_LOCAL_LIMITS (0, LIMIT)
        #include BOOST_PP_LOCAL_ITERATE()
        default:
            return &overflow;
    }
}
int main() {
    int i, j, k;
    std::cin >> i >> j >> k;
    bind(i, j, k)();
    return 0;
}
This works, but takes forever to compile--at least on Comeau C++.  The
reason is not the use of the pp-lib though (that is fast), it is because
of
the sheer amount of template instantiations.  E.g. you get 101
instantiations of the one-template-argument "bind," 10,201 instantiations
of
the two-template-argument "bind," and 1,030,301 instantiations of the
"target" function (+ 1,030,301 instantiations of "T").  Even if you use
and
the
...
"if/else if" statements (or nested switches), you still going to get
instantiations of "T" in the millions.  That means that you have to 1)
bite
the bullet and give up trying to do whatever you're doing with
compile-time
values or 2) seriously reduce the maximum value (to about 10, for
example).
In any case, the pp-lib can generate this, direct nested switches, or even
the permutations along the lines of "if/else if."  The problem is massive
instantiation and massive code bloat no matter how you do it.  IMHO, it
just
isn't worth it.
Another idea is to have an array of function pointers:
pbound_f array[101][101][101] = { ... };
...and initialize the elements with instantiations of some fully bound
function template (the pp-lib can, of course, do this as well).  And use
it
like this:
array[i][j][k]();
This will only avoid the first 10,302 instantiations though.  You are
still
going to get template instantiation in the millions.  What are you trying
to
do exactly?  Maybe there is a better way.
...
Naturally, I don't want to type all that in myself, so I was hoping the
boost preprocessor library could do it for me.  If someone knows of a way
to
do it, please let me know.
Regards,
Paul Mensonides
Info: http://www.boost.org
Wiki: http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl
Unsubscribe: mailto:boost-users-unsubscribe@yahoogroups.com
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/