1 $Id: TODO,v 1.26 1999-04-08 13:03:02 ivan Exp $
3 If you are interested in helping with any of these, please join the mailing
4 list (send a blank message to ivan-freeside-subscribe@sisd.com) to avoid
9 javascript (yuck!) "are you sure?" confirmation on cancelations, etc.
10 (view/cust_pkg and view/svc_*)
12 get rid of time2str("%D") which formats dates in a non-y2k-safe looking fashion
13 (all the actual date handling uses UNIX timestamps and is fine)
15 uncomment expire in view/cust_pkg.cgi and find the expire cron from fsold
17 Lay out any remaining ugly forms better.
19 remove "records identical" warning? gets in the way of more important stuff.
20 or fix logic which tries to update identical records??
21 1.2 should be quiet enough that the error log is useful, hopefully.
23 Postgres has a maximum column length of 31 characters (but see NAMEDATALEN in
24 postgres_ext.h). part_svc has columns like: svc_acct__radius_Attribute_flag
25 (22 characters!) It seems that stuff over the limit is silently ignored,
26 so we get 4 characters back. So, Radius_Attributes are max 13 characters with
27 stock Postgres. see rfc2138 for what's affected
28 What's a good fix? (besides recompiling postgres with NAMEDATALEN 64)
29 (mysql has a 64 character max column length. others?)
31 look at DBIx::Recordset! (and Tie::DBI, and...)
33 password and slipip stuff in svc_acct.pm store need to be split into two fields or something, so the silliness in svc_acct.pm and svc_acct.export with looking at the data to decide what to do with it can be fixed
35 i10n: Apache::Language
37 Apache::Session? Other useful Apache::* ?
39 email invoices are only sent for the BILL payby. If setup, should statements
40 (since they're not invoices) be sent for COMP and CARD as well?
42 $cgi->keywords is causing the (hard to trace) error:
43 Use of uninitialized value at (eval 5) line 5
45 edit/cust_main.cgi gives an uninformative error message:
46 > Can't call method "agentnum" without a package or object reference at
47 > /usr/local/apache-ssl/htdocs/freeside/edit/cust_main.cgi line 116.
48 if there are no agents.
50 (is this missing on any web screens? (easy with $cust_svc->label)
51 Add the ability for services to filter information up to the package level
52 for web screens, so you can select a particlar package based
53 on username or domain name, etc.
55 Allow a cancelled/suspended/active status from packages to bubble up to
56 the customer lists. Put active, then suspended, then cancelled accounts.
57 Similar ordering on the package listing inside a single customer.
59 false laziness: edit/cust_main.cgi got some parts copied from edit/svc_acct.cgi
60 the web interface in general needs to be redone in a more abstract way.
62 false laziness: some of search/svc_acct_sm.cgi was copied to search/svc_domain.cgi. but web interface in general needs to be rewritten in a mucho cleaner way.
64 Portability: in FS::Record, $sth->execute does not return a number of rows for all DBD's. see man DBI
66 subroutine the where clause (eventually all SQL) as OO perhaps (has anyone done this?)
68 add a select method to FS::Record?
70 one-time/per-customer/? changes in rates and descriptions ('remembered
71 invoices'): implement by creating a new package on the fly... but it isn't
72 associated with any agent types so it won't show up for other customers to buy.
73 (but also... make sure they go away when the customer does! - need this? :
74 one-off package edits! : need a cust_pkgs or cust_part_pkgs or something table,
75 with custnum and partpkg (like type_pkgs)
76 (what happens if you hit "custom pricing" but the pricing is already custom?)
78 You can't delete the stuff under administration yet. Add this,
79 _including_ making sure the thing you are deleting is not in use!
81 add links on view/cust_main.cgi to setup services, like view/cust_pkg.cgi
83 FS::cust_pkg _require_'s FS::$svc, but this won't work with %FS::UID::callback
84 loading of configuration. (pry need same idea, but will run immediately if
85 context allows). Looks like error is masked by 'use FS::cust_svc' which in
86 turn 'use's FS::{svc_acct, svc_acct_sm, svc_domain}' which is now explicit
89 Allow a cancelled/suspended/active status from packages to bubble up to
90 the customer lists. Put active, then suspended, then cancelled accounts.
91 Similar ordering on the package listing inside a single customer.
93 svc_domain.pm mail sending uses Date::Format which doesn't seem to pick up
96 view/svc_domain.cgi needs to know the domain might be unaudited (cosmetic)
98 remove whois_hack set to 1 for svc_domain.pm? add all known registries and
100 .us domains and others!
101 site_perl/svc_domain.cgi (hmm... or maybe should have a button? or maybe svc_domain.pm should handle this) should set $whois_hack for non-internic domains, so you can add them...
103 turn on the depriciation warnings for [e]idiot in FS::CGI. Stop using [e]idiot
104 the last places it is (htdocs/search/ htdocs/misc/ htdocs/misc/process)
106 (test cust_main.pm with cybercash v2 and v3, especially with the callback
107 stuff AND with mod_perl w/cybercash v2 kludge in package main)
108 (callback stuff should be eliminated by now)
110 bah, table/itable/*table in FS::CGI is silly.
112 doc Apache::AuthDBI as well
114 Provide sample httpd.conf files.
116 hey look: Tie::DBI! Check that out. Override its commit with something that
117 does perl-side caching for ? a performance improvement and as an emulation
118 layer to plug in f.ex mysql's atomic transactions
120 Record.pm uses does some non-portable DBI things. MySQL and Pg seem fine.
121 Fix it anyway unless we migrate to Tie::DBI.
125 cust_bill.pm uses '==' comparison on dates because they're currently ints
127 config file for allowed card types
129 write instructions for adding new services w/svc_Common.pm. Get rid of all
130 places where svc_* tables are hardcoded (rename svc_acct_pop to part_pop so
133 test and document libapache-dbi-logger (woo!)
135 radius logfile parsing and perl expression check.
137 Fix in cust_bill BUGS:
138 There is an off-by-one error in print_text which causes a visual error (Page 1
139 of 2 printed on some single-page invoices).
141 fields should be a method against a FS::Record or derived object, as well as
142 being something you can call as FS::Record::fields('tablename'). Might
143 even be able to handle both in the same routine (that would be neato).
145 Immediate removal of incorrectly entered check payments (can't take too
146 long to do this, or accounting is fubared).
148 Add code to move from one service to another (POP to SLIP/PPP, etc.).
149 This _should_ be possible by working off the rules in part_svc rather than
150 hardcoding anything in. The rules in part_svc may need some elaboration,
153 Use ut_ FS::Record methods in all derived classes (possibly some from dbdef?... eventually all from dbdef??? - but then `dbdef-create' would be impossible as there would be metadata we couldn't ask the backend for. hmm.)
155 (bring back from fsold, ) Generalize config-sending stuff and make more configurable.
156 Expand the HylaFAX interface (also possibly generalize for other fax
157 softwar ie .comfaxe); allow things like arbitrary faxes of sales
158 literature, specific troubleshooting documents and so on. Maybe even
159 allow users to do this (though that might not belong in Freeside).
161 misc/process/sendconfig.cgi
162 Configure fax recipients via a separate box rather than using the finger
163 name or first+last from cust_main.
165 move all phone number logic out of Freeside - let HylaFAX or whatever
168 soundex searches for customer name and company? where are free soundex tools? (standard Text::Soundex duh) - I could have sworn I saw Text::Soundex on CPAN?!
170 should be able to link on some field in email alias (right now you can link
171 on username or domain with a fallback to svcnum)
173 generalize and make configurable new invoice printing scheme in FS::cust_main::collect (past due)
175 deleting an svc_domain should delete all associated svc_acct_sm records.
176 same with a svc_acct.
178 periodic password encrypter
180 Automated, configurable notification, suspension and cancellation of
185 Allow for a future setup date on accounts.
187 sub AUTOLOAD in FS::Record should warn? die? if used with a non-existant column
190 edit (not just import, export and allow default/fixed) arbitrary radius stuff
192 edit/svc_acct.cgi and edit/process/svc_acct.cgi should deal with arbitrary radius stuff
194 radius import should take DEFAULT entry and put it in /var/spool/freeside/conf/radius-default ; svc_acct.export should use it (and doc)
196 in UI, s/State/State\/Provence/go and s/County/County\/Locality/go
198 what else (besides l10n) for i18n? (money!)
200 audit htdocs/* for things that should be libraried and things that should be
201 new methods on the objects (need to do this before implementing a new UI)
202 all the big things are done
204 some places we die() where we should &FS::CGI::idiot (and perhaps vice-versa).
205 Decide based on whether or not the "error" should show up in logs.
207 all .cgi's should use standard header/footer and idiot() subroutines. maybe HTML:: perl modules
208 for HTML creation. Maybe Embperl or something along those lines. ?
210 When running bin/bill, Fix this (Annoying but harmless):
211 Use of uninitialized value at /usr/local/lib/site_perl/FS/cust_pkg.pm line 99, <ADDRESS> chunk 4.
212 Use of uninitialized value at /usr/local/lib/site_perl/FS/cust_pkg.pm line 102, <ADDRESS> chunk 4.
213 Use of uninitialized value at /usr/local/lib/site_perl/FS/cust_pkg.pm line 105, <ADDRESS> chunk 4.
215 all cgi (but internal to the isp) places where package names are listed should also have
216 comment (like agent_type)
218 clean up $recref and other silliness and use -> calls where possible, or
219 one other alternative. clean up everything else.
220 should FS::Record use Tie::Hash? That would be very clean, but where do we
221 store the other information? Maybe you could ask any FS::Record object for a
224 change all file access from regular open(FILE,) stuff to OO, because of
225 problems scoping and passing filehandles like that.
227 the web interface should create a new object and use it instead of a blank
228 form for new records. the create method of svc_ objects should set defaults
231 sub check in man FS::table_name should be rewriteen. Get rid of $recref
232 stuff. Make sure all fields that refer to other database are checked.
234 Integration with signup disks (are there any free ones? Netscape?).
236 One-button cancel (+refund) for lusers who can't get online.
238 Keep information on virtual web servers (hostname, IP, host machine,
239 directory, etc.) and export this information for importation into the ISPs
242 Remove requirement that the first mail alias be the catchall? Still make
243 sure only one catchall per domain is defined in any case, of course.
245 Ability to move cust_pkg records from one customer to another? (proably
246 will need to cancel the old and create a new like when we move services
249 Auto-increment expired cards one year, and try again?
251 More non-US stuff - zip codes, country codes, foreign currencies, etc.
253 cust_refund.{cgi.pm} need to do cards xaxtions. (now we only have cust_credit)
255 Nicer set of integrated reporting possibilities, like weekly sales totals
256 by customer, package, agent, referral, etc., aging reports sorted by lots
257 of different things, and so on.
259 Client/server setup for users to modify their own passwords, shells, etc,
260 via passwd or secure web interface (prelminary passwd/chfn/chsh
261 replacement done). Complicated by the fact that we don't want to allow
262 incoming connections to the machine running Freeside, so we probably need
263 to have a daemon on each external shell or web machine that is contacted
264 by the Freeside machine. Be very very careful for both traditional
265 security issues and DoS problems.
267 An extension of the above to allow users to modify selected parts of their
268 own information, order and cancel services. A web interface for new
271 Expand domain name stuff to house all domain information. Export
272 named.boot/named.conf (primary and secondary) and named.{domain} files.
273 Add more registries (not just InterNIC's com org net edu)
275 Nice postscript paper invoices, rather than current ASCII invoices.
277 think about race-condititions in FS::Record and derived ->check ->insert
278 and so on, uid and username checks in svc_acct, etc.
280 Move to rsync over ssh file exportation rather than scp.
282 check 'n fix the proactive password checker. (cracklib?)
284 refunds of "BILL" payments: generate pseudo-check.
286 write batch senders and batch parsers for the different credit card processors
288 More CC processors/methods.
290 you should be able to fiddle the setup date in cust_pkg. (at least initially)
292 delete options in administration section
294 write a generic batch senders and batch parsers.
296 need a way to override svc_acct export on a per-machine basis; just use config files based on machine name i suppose; document that. (no, import desync_hosts
297 type stuff from cerkit)
299 add a table with column of export services (passwd, shadow, master.passwd, .qmail file update, dns update, etc.) and rows machine groups and whether or not to export that (and any necessary parameters). wasn't matt (vunderkid, not matt@michweb) working on this? find him? each machine goes in a group of its own as well as a group based on function. add a table with only svcpart and machine group. now, when you import from each machine, it can get its own accounts with one svcpart and universal accounts with another svcpart. (though that does make the username duplicate checking more interesting)
301 you should be able to get column types as a method against an FS::Record object
302 as well as dbdef->table($table)->column($column)->type
304 move to perl module for fuzzy and soundex searching.
306 package view needs to list extraneous services; we need to prevent the
307 creation of them so this never happens (and mark it as such in the source)
308 (the creation problem should be fixed - though they will still happen if people
309 fsck around in the data manually, so list them anyway)
311 add attribute dictionary to fs-setup as a menu, plus analyze users file to
314 Check for and report on duplicate billing accounts (cust_main, though many
315 will have a need for these so probably don't disable them outright.)
317 create a ->warn as well as a ->check method for all FS::table classes?
320 something to automate making a release and updating the web demo
322 export a debian-style (also redhat and?) /etc/group file aswell!
324 svc_acct_sm.import qmail import should pull in recipientmap people too.
326 .pm's like svc_acct.pm which need to do time-consuming things like ssh remotely
327 should fork and do them in a child.
329 i18n/l10n: take ALL messages and catalog them in english.txt or in database or something, so we can eventually go int'l. int'l currency support would be a help aswell.
331 get some of { city, county, state, zip } from the missing bits if
332 possible (where can i get the data to do this? usps.gov?)
334 additional interfaces (perltk? java?)
336 Put the GPL notice in all files.
338 integrate w/IDEA's signup server
340 $cust_bill->owed database field to be eliminated, replaced by a method call
341 that calculates on the fly. make sure to grep for ->(get|set)field('owed')
342 same for cust_credit->credited
344 Export quota information.
346 move all configuration to a central place. maybe in blob's in the
347 database. maybe even things like the code to execute when a username is
348 changed can be in there, so less of the distributed scripts change between
351 Implement setup and recurring fees as Safe perl expressions rather than
352 numbers, to allow for variable-rate services. Backwards compatibility is
353 obtained because { 43 } in perl is still 43. :) Define API to pass
354 starting and ending dates and any other necessary data to expression
355 (fees are currently evaluated as Safe expressions but more work needs to
356 be done to define an opmask for various needs, write examples
357 (usage-based billing, etc.) and so on).
359 Add the ability to modify the next billing date in cust_pkg, and take
360 appropriate action. This will allow the implementation of pro-rate/1st of
361 the month billing as well as the ability to manually fiddle with
362 anniversary dates in cust_pkg, so you can sync a customer's anniversary
363 date even if you're using anniversary billing (manually or automatically).
364 (now with above, we need to have a way to automatically pro-rate /^(\d+)$/
365 charges - anything more complicated should figure it out itself given
366 starting and ending dates [document that!])
368 Daily Radius log parsing into database; other logfile formats?
370 Callbacks to enforce hourly limits on accounts (suspend until the end of
371 the billing period?), for those who limit customers rather than tack on
374 Flag packages (part_pkg) as taxable or non/taxable as some ISPs (for
375 example) need to charge tax on equipment but not service (separate flags
376 for setup and recurring fee... or perhaps a setup_tax, setup_notax,
377 recur_tax and recur_notax fees, and possibly something more flexible if
380 Allow for a variable number of invoices for customers who need multiple
383 Add a mail alias service with table svc_acct (not domain mail aliasing
384 which is domain with svc_acct_sm)
386 (bring back from fsold) Change customer comment field from its current kludge to something more
389 Better work orders with more information. Should eventually open a ticket
390 when we have such a thing.
392 edit/process/svc_wo.cgi
393 Call tracking and trouble tickets.
395 More accoutability for complimentary accounts: approval, expiration, term
396 (no more than x months in advance) and notification.
397 Flag particular users (or all users, for that matter) as having their
398 passwords hidden and/or locked from users of Freeside (maybe need Freeside
399 security levels first?).
401 Better Freeside-level configurable access, for those ISP's who have
402 employees they can't trust. Right now you're "stuck" with setting up
403 .htaccess stuff yourself. This should really just be integrated.
405 configuration/setup should get web interface
407 /usr/local/etc/freeside should be configurable
409 (probably as part of some automated installation process?)
411 This requires some serious magic in FS::Record:
412 ok, if date_type in fs-setup is to be something besides int,
413 now we need to create wrappers
414 for them so they behave identically across RDBMS's, ie date pops out as as
415 UNIX timestamp (or an object of some sort? maybe even a blessed $obj which
416 is a string not a hashref for backwards compatibility?) and so on. (remember
417 to treat '0' as Not a Date instead of 1/1/70.
419 Add Freeside-level transactions for RDBMS's which don't support
420 transcations? (Currently we assume a minimal RDBMS which has no rollback,
421 transactions or atomic updates). Or just require a RDBMS that supports
422 rollback and/or atomic updates and get rid of the work-arounds? The /rdb
423 interface had this kludge on top of it but is a technical dead-end in most
424 other ways, unless it can gain an SQL parser and DBD interface.
426 if i'm really bored, find the /rdb interface in fsold and port it to NoSQL,
427 and while I'm add it add interfaces for AnyDBM_File tied hash.. hmm. Shouldn't
428 an FS::Record have something to do with a tied hash? But we don't want
429 performance to go gaga... maybe something with commit to help out here?
431 Ok: FS::Record gives you a tied hash, and you get methods for commit, etc.
433 Better automated comparison of our CC records with processors (CyberCash,
434 at least, has not always had 100% accuracy, though recent versions are
437 Expect or other pty based login check, where we actually connect to a
438 terminal server or shell machine and test logging in as the user (if we
439 are keeping a cleartext password for that user) (This is something tech
440 support often needs for new customers)
442 Use cust_main table for pre-sales tracking as well?
444 Automatic commision report and check generation via freq and prog (to
445 become a Safe perl expression) fields in agent table, and possibly others.
447 Database and add a mailed-out date and method for disk/CD mailing, so a
448 customer can call and you can say, "sent on xx/xx/xx via {US Mail, Fedex,
451 Inventory tracking for physical items such as routers (for sale or
452 lease... probably doesn't make a difference in the ordering... but if you
453 cancel a router lease the inventory should come back. hmm.)
455 -- Matt's wishlist ---
457 From matt@michweb.net Fri Feb 20 16:39:53 1998
458 Date: Thu, 19 Feb 1998 23:20:11 -0500
459 From: Matt Simerson <matt@michweb.net>
460 Reply-To: quadran-developer@netgoth.com
461 To: quadran-developer@netgoth.com
462 Subject: Re: Welcome to quadran-developer
464 >Whats it based on and what is it supposed to do? I'm interested, but
465 >unfortunatly, I don't have that much time to help on the project (I'm busily
466 >working on one of my own based around MySQL and Qt right now -- don't know
467 >if it will be GPL'ed or not yet -- we'll probably just use it in house since
468 >it is designed around our system)...
470 That's what I set out to find, but didn't find anything on the web site.
471 I'm looking for something that will do the following:
473 Single point of entry for users on a secure system:
474 Creates account on user (public) systems
475 update /etc/passwd/master.passwd file
476 update radius database (if necessary)
477 Set up up disk quotas (although I hacked adduser to do this)
478 Option for adding user to a mailing list(s)
479 Export of new user info to customizable report (for automated entry
481 accounting software, etc...)
484 Export credit card info for batch processing and have hooks built
485 in for other forms of electronic processing.
486 Batch-Payment (apply payments from formatted text file).
487 Customizable reports for manual entry/importing into Accounting
489 Email or laser print invoices
490 Sanity checks credit card numbers before processing (code available)
492 Simple method for disabling an account.
493 Arbitrary Expiration Dates (on a given day, in x days)
495 Changing password to '*'
496 Virtual customers disabling dns, http server, log processing, etc..
498 Billing for different account types:
499 Dialup monthly flat rate. Prorates for partial months.
500 Dialup monthly flat rate for x hours + hourly usage.
503 Virtual Web accounts - w/multiple mailboxes
505 Disk space used over quota.
506 Tech support minimum + hourly charges
507 Other for misc stuff (modem, RAM, etc...)
509 Per user definable RADIUS attributes (ties in with above)
512 IP filters (for dialup email only)
514 Keep logs of modem usage generated daily from radius accounting logs stored
515 on multiple radius servers.
517 Keep logs of disk usage generated from quota.
519 Method of adding virtual domains to your system:
520 Automatically grabs an IP address from a preassigned pool.
521 Creates a domain.com database file from database fields
522 Updates /etc/named.conf or /etc/named.boot and reloads named.
523 Add's virtual.com to /etc/sendmail.cw or qmail control files.
524 Edits your web servers httpd.conf file and restarts http server.
525 An optional section for adding vif's can be added if the users OS
526 supports adding them on the fly. Otherwise it's up to the end
527 user. Make a hook that can run a custom script that the user
528 tweaks for his system.
529 Update or create the config file your web stats analyzer needs. I've
530 done this for analog (free) and http-analyze. Probably
531 should only officially support analog and let users hack
532 it to their hearts desire.
533 I've already written scripts that do most of the virtual web stuff on my
534 system...in bash. Shouldn't be hard for a perlmeister to convert. In fact,
535 as long as all the info was stored in the database (username, domain name,
536 and ip pool) this could easily just be run as an external script that the
537 user tweaks to match his system.
539 We use a great accounting software (M.Y.O.B) that does all the AP, AR,
540 Payroll, Tax stuff, and most everything else we could need. It's already
541 set up for the type of checks we have, etc, etc... I just need something to
542 do the billing part. I can import/export sales and payments directly once
543 the billing part is done. You can't write accounting software as good as